2024-08-01T03:52:18.426 INFO:root:teuthology version: 0.0.1.dev379+g53ce146 2024-08-01T03:52:18.452 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-01T03:52:18.541 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7829148' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: f9911e17b12222fbe598fff73cd3ad10d20c5776 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: f9911e17b12222fbe598fff73cd3ad10d20c5776 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d99c9769be1521395979e8c23d9073adbf1ff43f owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4805 sha1: f9911e17b12222fbe598fff73cd3ad10d20c5776 sleep_before_teardown: 0 subset: 91/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d99c9769be1521395979e8c23d9073adbf1ff43f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdr2YJAkj1CnmW2/snwpSBmjLtuM6mvLIP9KjuZxWdmqIGOUGnhiGERgvwi/0tAVkI4gV3kZNYYT7OJ82pk23k= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTO+ChPSoJUvX8yUgCzxoKHpUOuLrm+zl9XbtpjmJPwn26W/qflwmTZWCVOF1eSUrkGMcmTkS7uU9wNzY5uxUo= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHon3wp8DZJKwSvPFFeqDwCI+YimuJpKYYCdEpFbM6mT58WMyzz+rlbewqhlS0EMbSMeYm/VDYr4r4bILPdXXd8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 53ce1462e129f6eb4071986336534c740fdebd31 timestamp: 2024-07-31_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-08-01T03:52:18.541 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa; will attempt to use it 2024-08-01T03:52:18.542 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks 2024-08-01T03:52:18.542 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-01T03:52:18.544 INFO:teuthology.task.internal:Checking packages... 2024-08-01T03:52:18.564 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f9911e17b12222fbe598fff73cd3ad10d20c5776' 2024-08-01T03:52:18.565 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-01T03:52:18.565 INFO:teuthology.packaging:ref: None 2024-08-01T03:52:18.565 INFO:teuthology.packaging:tag: None 2024-08-01T03:52:18.565 INFO:teuthology.packaging:branch: squid 2024-08-01T03:52:18.565 INFO:teuthology.packaging:sha1: f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T03:52:18.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-08-01T03:52:18.791 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1246.gd99c9769 2024-08-01T03:52:18.792 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-01T03:52:18.799 INFO:teuthology.task.internal:no buildpackages task found 2024-08-01T03:52:18.799 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-01T03:52:18.822 INFO:teuthology.task.internal:Saving configuration 2024-08-01T03:52:18.842 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-01T03:52:18.849 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-01T03:52:18.873 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-01 03:46:44.119070', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdr2YJAkj1CnmW2/snwpSBmjLtuM6mvLIP9KjuZxWdmqIGOUGnhiGERgvwi/0tAVkI4gV3kZNYYT7OJ82pk23k='} 2024-08-01T03:52:18.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-01 03:46:44.122056', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTO+ChPSoJUvX8yUgCzxoKHpUOuLrm+zl9XbtpjmJPwn26W/qflwmTZWCVOF1eSUrkGMcmTkS7uU9wNzY5uxUo='} 2024-08-01T03:52:18.917 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-01 03:46:44.120747', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHon3wp8DZJKwSvPFFeqDwCI+YimuJpKYYCdEpFbM6mT58WMyzz+rlbewqhlS0EMbSMeYm/VDYr4r4bILPdXXd8='} 2024-08-01T03:52:18.917 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-01T03:52:18.924 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-01T03:52:18.924 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-01T03:52:18.924 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-01T03:52:18.924 INFO:teuthology.run_tasks:Running task console_log... 2024-08-01T03:52:19.011 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcd8d4e6f80>, signals=[15]) 2024-08-01T03:52:19.012 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-01T03:52:19.018 INFO:teuthology.task.internal:Opening connections... 2024-08-01T03:52:19.018 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T03:52:19.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T03:52:19.098 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-08-01T03:52:19.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T03:52:19.185 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-08-01T03:52:19.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T03:52:19.262 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-01T03:52:19.269 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-08-01T03:52:19.288 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-08-01T03:52:19.288 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-08-01T03:52:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-08-01T03:52:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-08-01T03:52:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-08-01T03:52:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-08-01T03:52:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-01T03:52:19.347 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-01T03:52:19.348 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-08-01T03:52:19.384 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-08-01T03:52:19.402 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-08-01T03:52:19.402 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-08-01T03:52:19.460 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-01T03:52:19.461 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-01T03:52:19.461 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-08-01T03:52:19.484 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-08-01T03:52:19.501 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-08-01T03:52:19.501 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-08-01T03:52:19.559 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-01T03:52:19.560 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-01T03:52:19.560 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-08-01T03:52:19.583 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-01T03:52:19.593 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-01T03:52:19.597 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-01T03:52:19.598 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-08-01T03:52:19.600 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-08-01T03:52:19.603 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-08-01T03:52:19.618 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-01T03:52:19.624 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-01T03:52:19.624 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-08-01T03:52:19.657 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-08-01T03:52:19.659 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-08-01T03:52:19.763 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-01T03:52:19.836 INFO:teuthology.run_tasks:Running task kernel... 2024-08-01T03:52:19.851 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-01T03:52:19.851 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-08-01T03:52:19.852 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-08-01T03:52:19.852 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-01T03:52:19.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T03:52:19.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-01T03:52:20.047 DEBUG:teuthology.task.kernel:sha1 for client.0 is 34096973aec0bfbe2bb86f34701acc6cd4133fd9 2024-08-01T03:52:20.047 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "34096973aec0bfbe2bb86f34701acc6cd4133fd9"... 2024-08-01T03:52:20.047 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-08-01T03:52:20.065 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-437.el9.x86_64 2024-08-01T03:52:20.065 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 34096973aec0bfbe2bb86f34701acc6cd4133fd9 2024-08-01T03:52:20.066 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-01T03:52:20.066 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum upgrade -y linux-firmware 2024-08-01T03:52:22.235 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-08-01T03:52:25.553 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 20 MB 00:02 2024-08-01T03:52:30.794 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.6 MB 00:01 2024-08-01T03:52:32.913 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 18 kB 00:00 2024-08-01T03:52:33.907 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-01T03:52:39.730 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-08-01T03:52:41.939 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-08-01T03:52:41.953 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-08-01T03:52:41.953 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-08-01T03:52:41.953 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-08-01T03:52:41.953 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-08-01T03:52:41.953 INFO:teuthology.orchestra.run.smithi003.stdout: linux-firmware noarch 20240716-145.el9 baseos 383 M 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 1 Package 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 383 M 2024-08-01T03:52:41.954 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-08-01T03:52:53.696 INFO:teuthology.orchestra.run.smithi003.stdout:linux-firmware-20240716-145.el9.noarch.rpm 33 MB/s | 383 MB 00:11 2024-08-01T03:52:53.696 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-08-01T03:52:53.697 INFO:teuthology.orchestra.run.smithi003.stdout:Total 33 MB/s | 383 MB 00:11 2024-08-01T03:52:56.926 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-08-01T03:52:57.015 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-08-01T03:52:57.015 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-08-01T03:53:00.426 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-08-01T03:53:00.426 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-08-01T03:53:09.550 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-08-01T03:53:10.176 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : linux-firmware-20240716-145.el9.noarch 1/2 2024-08-01T03:53:10.266 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-08-01T03:53:11.026 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-08-01T03:53:11.026 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 1/2 2024-08-01T03:53:11.712 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-08-01T03:53:11.712 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T03:53:11.712 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-08-01T03:53:11.712 INFO:teuthology.orchestra.run.smithi003.stdout: linux-firmware-20240716-145.el9.noarch 2024-08-01T03:53:11.712 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T03:53:11.712 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-08-01T03:53:12.014 INFO:teuthology.task.kernel:Downloading kernel 34096973aec0bfbe2bb86f34701acc6cd4133fd9 on client.0... 2024-08-01T03:53:12.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=34096973aec0bfbe2bb86f34701acc6cd4133fd9 2024-08-01T03:53:12.162 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/34096973aec0bfbe2bb86f34701acc6cd4133fd9/centos/9/flavors/default/x86_64/ 2024-08-01T03:53:12.162 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.10.0_g34096973aec0-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/34096973aec0bfbe2bb86f34701acc6cd4133fd9/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-01T03:53:12.165 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi003.front.sepia.ceph.com to complete... 2024-08-01T03:53:18.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01 03:53:18 URL:https://1.chacra.ceph.com/r/kernel/testing/34096973aec0bfbe2bb86f34701acc6cd4133fd9/centos/9/flavors/default/x86_64/kernel-6.10.0_g34096973aec0-1.x86_64.rpm [358296803/358296803] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-01T03:53:18.429 INFO:teuthology.orchestra.run.smithi003.stderr:FINISHED --2024-08-01 03:53:18-- 2024-08-01T03:53:18.429 INFO:teuthology.orchestra.run.smithi003.stderr:Total wall clock time: 6.2s 2024-08-01T03:53:18.429 INFO:teuthology.orchestra.run.smithi003.stderr:Downloaded: 1 files, 342M in 6.0s (57.2 MB/s) 2024-08-01T03:53:18.432 INFO:teuthology.task.kernel:Installing kernel 34096973aec0bfbe2bb86f34701acc6cd4133fd9 on client.0... 2024-08-01T03:53:18.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-01T03:53:20.588 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying... ######################################## 2024-08-01T03:53:20.634 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing... ######################################## 2024-08-01T03:53:20.688 INFO:teuthology.orchestra.run.smithi003.stdout:Updating / installing... 2024-08-01T03:53:29.000 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-6.10.0_g34096973aec0-1 ######################################## 2024-08-01T03:54:09.437 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-01T03:54:09.437 DEBUG:teuthology.orchestra.run.smithi003:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/System.map-6.10.0-g34096973aec0 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/config-6.10.0-g34096973aec0 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/boot/vmlinuz-6.10.0-g34096973aec0 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/System.map 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/config 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel 2024-08-01T03:54:09.516 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-01T03:54:09.517 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kernel 2024-08-01T03:54:09.518 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kernel/cpu 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kernel/cpu/mce 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kvm 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/arch/x86/kvm/kvm.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/ansi_cprng.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/anubis.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/arc4.ko 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx 2024-08-01T03:54:09.519 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx/async_memcpy.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx/async_pq.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx/async_tx.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx/async_xor.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/async_tx/raid6test.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/authenc.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/authencesn.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/blake2b_generic.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/blowfish_common.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/blowfish_generic.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/camellia_generic.ko 2024-08-01T03:54:09.520 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/cast5_generic.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/cast6_generic.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/cast_common.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/ccm.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/cmac.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/crc32_generic.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/crypto_engine.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/crypto_user.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/cts.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/des_generic.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/dh_generic.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/ecc.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/ecdh_generic.ko 2024-08-01T03:54:09.521 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/echainiv.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/essiv.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/fcrypt.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/khazad.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/lz4.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/lz4hc.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/md4.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/michael_mic.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/pcbc.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/pcrypt.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/rmd160.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/seed.ko 2024-08-01T03:54:09.522 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/serpent_generic.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/tcrypt.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/tea.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/twofish_common.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/twofish_generic.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/vmac.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/wp512.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/xcbc.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/xor.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/crypto/xxhash_generic.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-01T03:54:09.523 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/acpi_pad.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/ec_sys.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/platform_profile.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/sbs.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/sbshc.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/acpi/video.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/acard-ahci.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/ahci_platform.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/ata_generic.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/libahci_platform.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_acpi.ko 2024-08-01T03:54:09.524 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_ali.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_amd.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_artop.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_atiixp.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_atp867x.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_cmd640.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_cmd64x.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_cypress.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_efar.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_hpt366.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_hpt37x.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-01T03:54:09.525 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_it8213.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_it821x.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_jmicron.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_marvell.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_mpiix.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_netcell.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_ninja32.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_ns87410.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_ns87415.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_oldpiix.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_opti.ko 2024-08-01T03:54:09.526 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_optidma.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_pcmcia.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_piccolo.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_rdc.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_sch.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_serverworks.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_sil680.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_sis.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_sl82c105.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_triflex.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pata_via.ko 2024-08-01T03:54:09.527 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/pdc_adma.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_inic162x.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_mv.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_nv.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_promise.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_qstor.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_sil.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_sil24.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_sis.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_svw.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_sx4.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_uli.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_via.ko 2024-08-01T03:54:09.528 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ata/sata_vsc.ko 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/auxdisplay 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/auxdisplay/ks0108.ko 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/base 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/base/regmap 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/bcma 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/bcma/bcma.ko 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/aoe 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/aoe/aoe.ko 2024-08-01T03:54:09.529 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/brd.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/drbd 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/drbd/drbd.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/floppy.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/loop.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/mtip32xx 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/nbd.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/null_blk 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/null_blk/null_blk.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/pktcdvd.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/rbd.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/virtio_blk.ko 2024-08-01T03:54:09.530 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/xen-blkback 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/xen-blkfront.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/zram 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/block/zram/zram.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hangcheck-timer.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hw_random 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hw_random/via-rng.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-01T03:54:09.531 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/lp.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/mwave 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/mwave/mwave.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/ppdev.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tlclk.ko 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm 2024-08-01T03:54:09.532 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm/tpm.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/uv_mmtimer.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/char/virtio_console.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-01T03:54:09.533 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/ccp 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/ccp/ccp.ko 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/intel 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/intel/qat 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/intel/qat/qat_common 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-01T03:54:09.534 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/padlock-aes.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/padlock-sha.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/virtio 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dca 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dca/dca.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/devfreq 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dma 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dma/dw 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dma/ioat 2024-08-01T03:54:09.535 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/amd64_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/e752x_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/edac_mce_amd.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i3000_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i3200_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i5100_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i5400_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i7300_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i7core_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/i82975x_edac.ko 2024-08-01T03:54:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/ie31200_edac.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/sb_edac.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/edac/x38_edac.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/firmware 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/firmware/edd.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/gpio 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/gpio/gpio-ich.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-appleir.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-aureal.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-axff.ko 2024-08-01T03:54:09.537 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-dr.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-elecom.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-elo.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-emsff.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-gaff.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-gt683r.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-gyration.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-holtekff.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-hyperv.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-icade.ko 2024-08-01T03:54:09.538 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-keytouch.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-kye.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-lcpower.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-led.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-lenovo.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-multitouch.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-ortek.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-penmount.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-petalynx.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-picolcd.ko 2024-08-01T03:54:09.539 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-pl.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-plantronics.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-primax.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-rmi.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-common.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-01T03:54:09.540 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-roccat.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-saitek.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-samsung.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-sjoy.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-sony.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-speedlink.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-steelseries.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-sunplus.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-tivo.ko 2024-08-01T03:54:09.541 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-tmff.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-topseed.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-twinhan.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-uclogic.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-waltop.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-wiimote.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-xinmo.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-zpff.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/hid-zydacron.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/uhid.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hid/wacom.ko 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hv 2024-08-01T03:54:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hv/hv_balloon.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hv/hv_utils.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hv/hv_vmbus.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/abituguru.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/abituguru3.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ad7414.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ad7418.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adc128d818.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adm1025.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adm1026.ko 2024-08-01T03:54:09.543 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adm1029.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adm1031.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adm9240.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ads7828.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adt7410.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adt7411.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adt7462.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adt7470.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adt7475.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/adt7x10.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/amc6821.ko 2024-08-01T03:54:09.544 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/applesmc.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/asb100.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/asc7621.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/atxp1.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/coretemp.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/dme1737.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ds1621.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ds620.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/emc1403.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/emc6w201.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/f71805f.ko 2024-08-01T03:54:09.545 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/f71882fg.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/f75375s.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/fam15h_power.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/fschmd.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/g760a.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/g762.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/gl518sm.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/gl520sm.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/i5500_temp.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/i5k_amb.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ibmaem.ko 2024-08-01T03:54:09.546 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ibmpex.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ina209.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ina2xx.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/it87.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/k10temp.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/k8temp.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lineage-pem.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm63.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm73.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm75.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm77.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm78.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm80.ko 2024-08-01T03:54:09.547 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm83.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm85.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm87.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm90.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm92.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm93.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm95234.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm95241.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/lm95245.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc2945.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc4151.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc4215.ko 2024-08-01T03:54:09.548 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc4222.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc4245.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc4260.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/ltc4261.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max16065.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max1619.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max1668.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max197.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max6639.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max6650.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/max6697.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/mcp3021.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/nct6683.ko 2024-08-01T03:54:09.549 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/nct6775-core.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/nct6775.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/nct7802.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pc87360.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pc87427.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pcf8591.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-01T03:54:09.550 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/powr1220.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/sch5627.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/sch5636.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/sht15.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/sht21.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/shtc1.ko 2024-08-01T03:54:09.551 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/sis5595.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/smsc47b397.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/smsc47m1.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/smsc47m192.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/thmc50.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/tmp102.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/tmp103.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/tmp401.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/tmp421.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/via-cputemp.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/via686a.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/vt1211.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/vt8231.ko 2024-08-01T03:54:09.552 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83627ehf.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83627hf.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83781d.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83791d.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83792d.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83793.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83795.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83l785ts.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/hwmon/w83l786ng.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/algos 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-01T03:54:09.553 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-01T03:54:09.554 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/i2c-dev.ko 2024-08-01T03:54:09.555 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/i2c-mux.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/i2c-smbus.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/i2c/i2c-stub.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/ib_core.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-01T03:54:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/cxgb4 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/mlx4 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/mlx5 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/mthca 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/ipoib 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-01T03:54:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/iser 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/isert 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/srp 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/srpt 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/gameport 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-01T03:54:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/gameport/gameport.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/joydev.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/keyboard 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/appletouch.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/bcm5974.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/cyapatp.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/sermouse.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-01T03:54:09.559 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/rmi4 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/serio 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/serio/altera_ps2.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/serio/arc_ps2.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/serio/serio_raw.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/input/sparse-keymap.ko 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds 2024-08-01T03:54:09.560 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/leds-blinkm.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/leds-lm3530.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/leds-lp3944.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/leds-ss4200.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-01T03:54:09.561 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/bcache 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/bcache/bcache.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-bio-prison.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-cache-smq.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-cache.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-crypt.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-delay.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-flakey.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-log-userspace.ko 2024-08-01T03:54:09.562 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-multipath.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-queue-length.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-raid.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-round-robin.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-service-time.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-switch.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-thin-pool.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/dm-verity.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/persistent-data 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/raid0.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/raid1.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/raid10.ko 2024-08-01T03:54:09.563 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/md/raid456.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptbase.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptctl.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptfc.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptlan.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptsas.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptscsih.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/message/fusion/mptspi.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/lpc_ich.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/lpc_sch.ko 2024-08-01T03:54:09.564 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/mfd-core.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/sm501.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/viperboard.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/vx855.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/mfd/wl1273-core.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/altera-stapl 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/apds9802als.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/apds990x.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/bh1770glc.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/cb710 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/cb710/cb710.ko 2024-08-01T03:54:09.565 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/eeprom 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/eeprom/at24.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/eeprom/max6875.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/enclosure.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/hpilo.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/isl29003.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/isl29020.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/lis3lv02d 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/mei 2024-08-01T03:54:09.566 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/mei/mei-me.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/mei/mei-txe.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/mei/mei.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/sgi-gru 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/sgi-xp 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/tifm_7xx1.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/tifm_core.ko 2024-08-01T03:54:09.567 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/tsl2550.ko 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/vmw_balloon.ko 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/vmw_vmci 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/bonding 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/bonding/bonding.ko 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa/b53 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-01T03:54:09.568 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa/mv88e6xxx 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/dummy.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/eql.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/3com 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/8390 2024-08-01T03:54:09.569 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/adaptec 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/agere 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/alteon 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/altera 2024-08-01T03:54:09.570 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/amd 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/alx 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atlx 2024-08-01T03:54:09.571 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/genet 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/brocade 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/brocade/bna 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-01T03:54:09.572 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/cisco 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/cisco/enic 2024-08-01T03:54:09.573 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dlink 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-01T03:54:09.574 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/dnet.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/emulex 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/emulex/benet 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/ethoc.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/fealnx.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/e1000 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/e1000e 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-01T03:54:09.575 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/fm10k 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/i40e 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/iavf 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/igb 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/igbvf 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-01T03:54:09.576 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/libeth 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/libie 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/jme.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/marvell 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-01T03:54:09.577 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/micrel 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/myricom 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/natsemi 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-01T03:54:09.578 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/neterion 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/nvidia 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/packetengines 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-01T03:54:09.579 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/qlogic 2024-08-01T03:54:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-01T03:54:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-01T03:54:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-01T03:54:09.581 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/rdc 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/realtek 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/rocker 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sfc 2024-08-01T03:54:09.582 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/silan 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sis 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/smsc 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/stmicro 2024-08-01T03:54:09.583 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sun 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/tehuti 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/ti 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/via 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-01T03:54:09.584 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/wiznet 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/xircom 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/geneve.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/hyperv 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ieee802154 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ifb.ko 2024-08-01T03:54:09.585 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ipvlan 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/macvlan.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/macvtap.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/mdio 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/mdio.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/mii.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/net_failover.ko 2024-08-01T03:54:09.586 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/netconsole.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/nlmon.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/pcs 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/amd.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/bcm87xx.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/broadcom.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/cicada.ko 2024-08-01T03:54:09.587 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/davicom.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/dp83640.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/et1011c.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/icplus.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/lxt.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/marvell.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/micrel.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/national.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/phylink.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/qsemi.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/realtek.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/smsc.ko 2024-08-01T03:54:09.588 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/ste10Xp.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/phy/vitesse.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/ppp_async.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/pppoe.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/pppox.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/ppp/pptp.ko 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/slip 2024-08-01T03:54:09.589 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/slip/slhc.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/slip/slip.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/sungem_phy.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/tap.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team/team.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team/team_mode_random.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/tun.ko 2024-08-01T03:54:09.590 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/veth.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/virtio_net.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/vmxnet3 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/vxlan 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/vxlan/vxlan.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/xen-netback 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/net/xen-netfront.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ntb 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ntb/ntb.ko 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/nvme 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/nvme/host 2024-08-01T03:54:09.591 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/nvme/host/nvme-core.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/nvme/host/nvme.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/parport 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/parport/parport.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/parport/parport_cs.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/parport/parport_pc.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/parport/parport_serial.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pci 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pci/hotplug 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pci/pcie 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pci/xen-pcifront.ko 2024-08-01T03:54:09.592 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pcmcia 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pcmcia/i82092.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pcmcia/pd6729.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pinctrl 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pinctrl/intel 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/acerhdf.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-01T03:54:09.593 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/hdaps.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/intel 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/intel_ips.ko 2024-08-01T03:54:09.594 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/platform/x86/wmi.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/powercap 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-01T03:54:09.595 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pps 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pps/clients 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pps/clients/pps_parport.ko 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/pps/pps_core.ko 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ptp 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ptp/ptp.ko 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ptp/ptp_kvm.ko 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ras 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ras/amd 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ras/amd/atl 2024-08-01T03:54:09.596 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ras/amd/fmpm.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-01T03:54:09.597 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-em3027.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-max6900.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-01T03:54:09.598 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/rtc/rtc-x1205.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/3w-9xxx.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/3w-sas.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/3w-xxxx.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/BusLogic.ko 2024-08-01T03:54:09.599 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/a100u2w.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/aacraid 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/advansys.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/aic7xxx 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/am53c974.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/arcmsr 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/atp870u.ko 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/be2iscsi 2024-08-01T03:54:09.600 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/bfa 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/bfa/bfa.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/bnx2fc 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/bnx2i 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/ch.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/csiostor 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/cxgbi 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-01T03:54:09.601 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/dc395x.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/device_handler 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/dmx3191d.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/esas2r 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-01T03:54:09.602 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/esp_scsi.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/fcoe 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/fnic 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/fnic/fnic.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/hpsa.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/hptiop.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/hv_storvsc.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/initio.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/ipr.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/ips.ko 2024-08-01T03:54:09.603 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/isci 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/isci/isci.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/libfc 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/libfc/libfc.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/libiscsi.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/libsas 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/libsas/libsas.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/lpfc 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/megaraid 2024-08-01T03:54:09.604 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/megaraid.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/mpt3sas 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/mvsas 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/mvumi.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/pm8001 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/pmcraid.ko 2024-08-01T03:54:09.605 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/qla1280.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/qla2xxx 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/qla4xxx 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/raid_class.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/scsi_debug.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-01T03:54:09.606 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/ses.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/st.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/stex.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/sym53c8xx_2 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/virtio_scsi.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/scsi/wd719x.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ssb 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ssb/ssb.ko 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/iscsi 2024-08-01T03:54:09.607 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/loopback 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/target_core_file.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/target_core_iblock.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/target_core_mod.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/target_core_pscsi.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/target_core_user.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/tcm_fc 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal 2024-08-01T03:54:09.608 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-01T03:54:09.609 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/ipwireless 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/n_gsm.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/n_hdlc.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/nozomi.ko 2024-08-01T03:54:09.610 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/serial 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/serial/8250 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/serial/arc_uart.ko 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/serial/jsm 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ufs 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ufs/core 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ufs/host 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/uio 2024-08-01T03:54:09.611 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/uio/uio.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/uio/uio_aec.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/uio/uio_cif.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/uio/uio_pci_generic.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/uio/uio_sercos3.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/class 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/class/cdc-acm.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/class/usblp.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/class/usbtmc.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/host 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/image 2024-08-01T03:54:09.612 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/image/mdc800.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/image/microtek.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/adutux.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/appledisplay.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/emi26.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/emi62.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/ezusb.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/idmouse.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/iowarrior.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/ldusb.ko 2024-08-01T03:54:09.613 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/legousbtower.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/sisusbvga 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/usb3503.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/usblcd.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/uss720.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/misc/yurex.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/phy 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/phy/phy-generic.ko 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial 2024-08-01T03:54:09.614 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/aircable.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ark3116.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ch341.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/cp210x.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/cyberjack.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/empeg.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-01T03:54:09.615 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/io_ti.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ipaq.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ipw.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ir-usb.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/keyspan.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/mct_u232.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/mos7720.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/mos7840.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/navman.ko 2024-08-01T03:54:09.616 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/omninet.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/opticon.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/option.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/oti6858.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/pl2303.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/qcaux.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/qcserial.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/quatech2.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/safe_serial.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/sierra.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ssu100.ko 2024-08-01T03:54:09.617 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/symbolserial.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/usb_debug.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/visor.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/whiteheat.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/uas.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-01T03:54:09.618 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-karma.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/storage/usb-storage.ko 2024-08-01T03:54:09.619 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/usbip 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vfio 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vfio/pci 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vfio/vfio.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vhost 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vhost/vhost.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-01T03:54:09.620 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vhost/vhost_net.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/vhost/vhost_scsi.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/video 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/video/backlight 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/video/backlight/apple_bl.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/video/backlight/lcd.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio_balloon.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio_mmio.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio_pci.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-01T03:54:09.621 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/virtio/virtio_ring.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/masters 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/masters/ds2482.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/masters/ds2490.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-01T03:54:09.622 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/w1/wire.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/hpwdt.ko 2024-08-01T03:54:09.623 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/i6300esb.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/ib700wdt.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/ibmasr.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/it87_wdt.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/machzwd.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/nv_tco.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-01T03:54:09.624 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/softdog.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/via_wdt.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/wdt_pci.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/watchdog/xen_wdt.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-evtchn.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-gntalloc.ko 2024-08-01T03:54:09.625 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-gntdev.ko 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-pciback 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xen-privcmd.ko 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xenfs 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/9p 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/9p/9p.ko 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/affs 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/affs/affs.ko 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/befs 2024-08-01T03:54:09.626 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/befs/befs.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/binfmt_misc.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/btrfs 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/btrfs/btrfs.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/cachefiles 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/cachefiles/cachefiles.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ceph 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ceph/ceph.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/coda 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/coda/coda.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/cramfs 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/cramfs/cramfs.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/dlm 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/dlm/dlm.ko 2024-08-01T03:54:09.627 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ecryptfs 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/f2fs 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/f2fs/f2fs.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fat 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fat/fat.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fat/msdos.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fat/vfat.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fuse 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fuse/cuse.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/fuse/fuse.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/gfs2 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/gfs2/gfs2.ko 2024-08-01T03:54:09.628 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/hfs 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/hfs/hfs.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/hfsplus 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/hfsplus/hfsplus.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/isofs 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/isofs/isofs.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/jfs 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/jfs/jfs.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/lockd 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/lockd/lockd.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/minix 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/minix/minix.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/netfs 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/netfs/netfs.ko 2024-08-01T03:54:09.629 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/blocklayout 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/filelayout 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/flexfilelayout 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/nfs.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/nfsv3.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs/nfsv4.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs_common 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs_common/grace.ko 2024-08-01T03:54:09.630 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfs_common/nfs_acl.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfsd 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nfsd/nfsd.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nilfs2 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nilfs2/nilfs2.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-celtic.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-centeuro.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-croatian.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-cyrillic.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-gaelic.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-greek.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-iceland.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-inuit.ko 2024-08-01T03:54:09.631 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-roman.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-romanian.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/mac-turkish.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp1250.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp1251.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp1255.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp737.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp775.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp850.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp852.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp855.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp857.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp860.ko 2024-08-01T03:54:09.632 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp861.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp862.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp863.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp864.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp865.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp866.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp869.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp874.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp932.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp936.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp949.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_cp950.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_euc-jp.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-1.ko 2024-08-01T03:54:09.633 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-13.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-14.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-15.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-2.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-3.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-4.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-5.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-6.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-7.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_iso8859-9.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_koi8-r.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_koi8-ru.ko 2024-08-01T03:54:09.634 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_koi8-u.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/nls/nls_utf8.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/cluster 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/dlm 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/dlmfs 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/ocfs2.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-01T03:54:09.635 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/overlayfs 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/overlayfs/overlay.ko 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/pstore 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/pstore/ramoops.ko 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/reiserfs 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/reiserfs/reiserfs.ko 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/romfs 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/romfs/romfs.ko 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/smb 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/smb/client 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/smb/client/cifs.ko 2024-08-01T03:54:09.636 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/smb/common 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/smb/common/cifs_arc4.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/smb/common/cifs_md4.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/squashfs 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/squashfs/squashfs.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/sysv 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/sysv/sysv.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/udf 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/udf/udf.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ufs 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/ufs/ufs.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/xfs 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/fs/xfs/xfs.ko 2024-08-01T03:54:09.637 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/kernel 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/kernel/rcu 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/kernel/rcu/rcutorture.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/kernel/torture.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/kernel/trace 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/asn1_encoder.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/crc-itu-t.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/crc8.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/crypto 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/crypto/libaescfb.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/crypto/libarc4.ko 2024-08-01T03:54:09.638 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/crypto/libdes.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/dim 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/dim/dimlib.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/libcrc32c.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/lru_cache.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/lz4 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/lz4/lz4_compress.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/lz4/lz4hc_compress.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/math 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/math/cordic.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/raid6 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/raid6/raid6_pq.ko 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/reed_solomon 2024-08-01T03:54:09.639 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/ts_bm.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/ts_fsm.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/ts_kmp.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/zstd 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/lib/zstd/zstd_compress.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/mm 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/mm/hwpoison-inject.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/6lowpan.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_dest.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_fragment.ko 2024-08-01T03:54:09.640 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_hop.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_mobility.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_routing.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/6lowpan/nhc_udp.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/802 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/802/garp.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/802/mrp.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/802/p8022.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/802/psnap.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/802/stp.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/8021q 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/8021q/8021q.ko 2024-08-01T03:54:09.641 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/9p 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/9p/9pnet.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/9p/9pnet_fd.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/9p/9pnet_rdma.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/9p/9pnet_virtio.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/batman-adv 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/batman-adv/batman-adv.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/br_netfilter.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/bridge.ko 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter 2024-08-01T03:54:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-01T03:54:09.643 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/ebtables.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ceph 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ceph/libceph.ko 2024-08-01T03:54:09.644 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/core 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/core/failover.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/core/pktgen.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dccp 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dccp/dccp.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dccp/dccp_diag.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dccp/dccp_ipv4.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dccp/dccp_ipv6.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dns_resolver 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dns_resolver/dns_resolver.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dsa 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dsa/dsa_core.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dsa/tag_brcm.ko 2024-08-01T03:54:09.645 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dsa/tag_dsa.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dsa/tag_none.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/dsa/tag_trailer.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ieee802154 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ieee802154/6lowpan 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ieee802154/ieee802154.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/ah4.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/esp4.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/fou.ko 2024-08-01T03:54:09.646 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/gre.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/inet_diag.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/ip_gre.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/ip_tunnel.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/ip_vti.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/ipcomp.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/ipip.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-01T03:54:09.647 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-01T03:54:09.648 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_bic.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_dctcp.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_diag.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_highspeed.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_htcp.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_hybla.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_illinois.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_lp.ko 2024-08-01T03:54:09.649 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_scalable.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_vegas.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_veno.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_westwood.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tcp_yeah.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/tunnel4.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/udp_diag.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/udp_tunnel.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/ah6.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/esp6.ko 2024-08-01T03:54:09.650 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/fou6.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/ip6_tunnel.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/ip6_vti.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/ipcomp6.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-01T03:54:09.651 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-01T03:54:09.652 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/sit.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/tunnel6.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/key 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/key/af_key.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_core.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_eth.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_ip.ko 2024-08-01T03:54:09.653 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_ip6.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_netlink.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/l2tp/l2tp_ppp.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/llc 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/llc/llc.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/mac802154 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/mac802154/mac802154.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/mpls 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/mpls/mpls_gso.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-01T03:54:09.654 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-01T03:54:09.655 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-01T03:54:09.656 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conncount.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-01T03:54:09.657 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_flow_table.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_log_syslog.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_nat.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_nat_irc.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_nat_sip.ko 2024-08-01T03:54:09.658 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nf_tables.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nfnetlink.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nfnetlink_log.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_chain_nat.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_compat.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_ct.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_fib.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-01T03:54:09.659 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_hash.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_limit.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_log.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_masq.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_nat.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_numgen.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_queue.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_quota.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_redir.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_reject.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_reject_inet.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_socket.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/nft_tunnel.ko 2024-08-01T03:54:09.660 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_AUDIT.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_CT.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_DSCP.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_HL.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_HMARK.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_LED.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_LOG.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_NETMAP.ko 2024-08-01T03:54:09.661 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_NFLOG.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_RATEEST.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_SECMARK.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_TEE.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_TPROXY.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_TRACE.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_addrtype.ko 2024-08-01T03:54:09.662 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_bpf.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_cgroup.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_cluster.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_comment.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_connbytes.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_connlabel.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_connlimit.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_connmark.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_conntrack.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_cpu.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_dccp.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_devgroup.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_dscp.ko 2024-08-01T03:54:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_ecn.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_esp.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_hashlimit.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_helper.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_hl.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_ipcomp.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_iprange.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_ipvs.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_l2tp.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_length.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_limit.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_mac.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_mark.ko 2024-08-01T03:54:09.664 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_multiport.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_nat.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_nfacct.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_osf.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_owner.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_physdev.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_pkttype.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_policy.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_quota.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_rateest.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_realm.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_recent.ko 2024-08-01T03:54:09.665 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_sctp.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_set.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_socket.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_state.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_statistic.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_string.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_tcpmss.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_time.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netfilter/xt_u32.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netlink 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/netlink/netlink_diag.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/nsh 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/nsh/nsh.ko 2024-08-01T03:54:09.666 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/openvswitch 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/openvswitch/openvswitch.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/openvswitch/vport-geneve.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/openvswitch/vport-gre.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/openvswitch/vport-vxlan.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/packet 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/packet/af_packet_diag.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/rds 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/rds/rds.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/rds/rds_rdma.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/rds/rds_tcp.ko 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched 2024-08-01T03:54:09.667 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_csum.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_gact.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_mirred.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_nat.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_pedit.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_police.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_simple.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_skbedit.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/act_vlan.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/cls_basic.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/cls_bpf.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/cls_flow.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/cls_fw.ko 2024-08-01T03:54:09.668 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/cls_route.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/cls_u32.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/em_cmp.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/em_ipset.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/em_meta.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/em_nbyte.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/em_text.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/em_u32.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_choke.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_codel.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_drr.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_fq.ko 2024-08-01T03:54:09.669 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_gred.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_hfsc.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_hhf.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_htb.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_ingress.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_mqprio.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_mqprio_lib.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_multiq.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_netem.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_pie.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_plug.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_prio.ko 2024-08-01T03:54:09.670 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_qfq.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_red.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_sfb.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_sfq.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_tbf.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sched/sch_teql.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sctp 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sctp/sctp.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sctp/sctp_diag.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc/auth_gss 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-01T03:54:09.671 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc/sunrpc.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc/xprtrdma 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/tipc 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/tipc/tipc.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/tipc/tipc_diag.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/unix 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/unix/unix_diag.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/vmw_vsock 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/vmw_vsock/vsock.ko 2024-08-01T03:54:09.672 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/xfrm 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/security 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/security/keys 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/security/keys/encrypted-keys 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/security/keys/trusted-keys 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/kernel/security/keys/trusted-keys/trusted.ko 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.alias 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.alias.bin 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.builtin 2024-08-01T03:54:09.673 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.builtin.alias.bin 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.builtin.bin 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.builtin.modinfo 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.dep 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.dep.bin 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.devname 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.order 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.softdep 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.symbols 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/modules.symbols.bin 2024-08-01T03:54:09.674 INFO:teuthology.orchestra.run.smithi003.stdout:/lib/modules/6.10.0-g34096973aec0/vmlinuz 2024-08-01T03:54:09.678 DEBUG:teuthology.task.kernel:get_image_version: 6.10.0-g34096973aec0 2024-08-01T03:54:09.678 DEBUG:teuthology.orchestra.run.smithi003:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-01T03:54:09.702 INFO:teuthology.orchestra.run.smithi003.stdout:postinstall scriptlet (using /bin/sh): 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout: /usr/bin/kernel-install add 6.10.0-g34096973aec0 /lib/modules/6.10.0-g34096973aec0/vmlinuz 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout:for file in vmlinuz System.map config; do 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout: if ! cmp --silent "/lib/modules/6.10.0-g34096973aec0/${file}" "/boot/${file}-6.10.0-g34096973aec0"; then 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout: cp "/lib/modules/6.10.0-g34096973aec0/${file}" "/boot/${file}-6.10.0-g34096973aec0" 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout: fi 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout:done 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -d "/lib/modules/6.10.0-g34096973aec0/dtb" ] && \ 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout: ! diff -rq "/lib/modules/6.10.0-g34096973aec0/dtb" "/boot/dtb-6.10.0-g34096973aec0" >/dev/null 2>&1; then 2024-08-01T03:54:09.703 INFO:teuthology.orchestra.run.smithi003.stdout: rm -rf "/boot/dtb-6.10.0-g34096973aec0" 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout: cp -r "/lib/modules/6.10.0-g34096973aec0/dtb" "/boot/dtb-6.10.0-g34096973aec0" 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:if [ ! -e "/lib/modules/6.10.0-g34096973aec0/modules.dep" ]; then 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout: /usr/sbin/depmod 6.10.0-g34096973aec0 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-install remove 6.10.0-g34096973aec0 2024-08-01T03:54:09.704 INFO:teuthology.orchestra.run.smithi003.stdout:fi 2024-08-01T03:54:09.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 68.el9 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8264254 2024-08-01T03:54:09.781 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-01T03:54:09.782 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-01T03:54:09.783 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-08-01T03:54:09.783 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T03:54:09.783 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-08-01T03:54:09.786 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-01T03:54:09.786 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 6.10.0-g34096973aec0 2024-08-01T03:54:09.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-01T03:54:10.432 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-08-01T03:54:11.951 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-01T03:54:11.969 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-08-01T03:54:11.973 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-08-01T03:54:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.UpMuHZvrRE 2024-08-01T03:54:11.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UpMuHZvrRE 2024-08-01T03:54:12.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.UpMuHZvrRE 2024-08-01T03:54:12.196 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.UpMuHZvrRE is 6KB 2024-08-01T03:54:12.245 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.UpMuHZvrRE 2024-08-01T03:54:12.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-08-01T03:54:12.323 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-08-01T03:54:12.323 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-08-01T03:54:12.324 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-08-01T03:54:12.324 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.10.0-g34096973aec0.conf 2024-08-01T03:54:12.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.10.0-g34096973aec0 2024-08-01T03:54:12.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-08-01T03:54:42.470 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-01T03:54:42.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-08-01T03:54:42.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T03:55:00.958 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-08-01T03:55:09.960 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-08-01T03:55:09.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T03:56:09.963 DEBUG:teuthology.orchestra.remote:timed out 2024-08-01T03:56:21.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-08-01T03:56:21.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T03:56:22.362 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-08-01T03:56:23.041 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-08-01T03:56:23.041 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.10.0-g34096973aec0"... 2024-08-01T03:56:23.041 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-08-01T03:56:23.095 INFO:teuthology.orchestra.run.smithi003.stdout:6.10.0-g34096973aec0 2024-08-01T03:56:23.095 DEBUG:teuthology.task.kernel:current kernel version is 6.10.0-g34096973aec0 vs 6.10.0-g34096973aec0 2024-08-01T03:56:23.095 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-01T03:56:23.095 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-01T03:56:24.096 DEBUG:teuthology.parallel:result is None 2024-08-01T03:56:24.096 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-01T03:56:24.104 INFO:teuthology.task.internal:Creating test directory... 2024-08-01T03:56:24.104 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-01T03:56:24.111 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-01T03:56:24.114 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-01T03:56:24.166 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-01T03:56:24.172 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-01T03:56:24.180 INFO:teuthology.task.internal:Creating archive directory... 2024-08-01T03:56:24.180 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-01T03:56:24.187 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-01T03:56:24.190 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-01T03:56:24.303 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-01T03:56:24.310 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-01T03:56:24.310 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-01T03:56:24.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T03:56:24.327 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-01T03:56:24.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T03:56:24.345 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-01T03:56:24.352 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-01T03:56:24.369 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-01T03:56:24.414 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-01T03:56:24.427 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-01T03:56:24.433 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-01T03:56:24.456 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-01T03:56:24.481 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-01T03:56:24.530 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-01T03:56:24.535 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-01T03:56:24.542 INFO:teuthology.task.internal:Configuring sudo... 2024-08-01T03:56:24.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-01T03:56:24.550 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-01T03:56:24.552 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-01T03:56:24.624 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-01T03:56:24.633 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-01T03:56:24.634 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-01T03:56:24.638 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-01T03:56:24.640 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-01T03:56:24.682 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-01T03:56:24.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-01T03:56:24.849 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-01T03:56:24.933 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-01T03:56:25.026 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T03:56:25.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-01T03:56:25.112 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-01T03:56:25.155 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-01T03:56:25.211 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-01T03:56:25.295 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-01T03:56:25.336 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T03:56:25.336 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-01T03:56:25.401 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-01T03:56:25.446 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-01T03:56:25.501 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-01T03:56:25.586 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-01T03:56:25.628 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T03:56:25.629 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-01T03:56:25.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-08-01T03:56:25.701 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-08-01T03:56:25.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-08-01T03:56:25.739 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-01T03:56:25.773 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-01T03:56:25.808 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-01T03:56:26.190 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-01T03:56:26.197 INFO:teuthology.task.internal:Starting timer... 2024-08-01T03:56:26.198 INFO:teuthology.run_tasks:Running task pcp... 2024-08-01T03:56:26.214 INFO:teuthology.run_tasks:Running task selinux... 2024-08-01T03:56:26.223 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-08-01T03:56:26.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-08-01T03:56:26.343 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-08-01T03:56:26.347 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-08-01T03:56:26.414 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-08-01T03:56:26.416 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-08-01T03:56:26.477 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-08-01T03:56:26.479 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-01T03:56:26.479 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-08-01T03:56:26.544 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-08-01T03:56:26.545 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-08-01T03:56:26.564 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-08-01T03:56:26.565 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-08-01T03:56:26.590 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-08-01T03:56:26.591 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-08-01T03:56:26.591 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-01T03:56:26.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T03:56:26.660 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-01T03:56:26.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T03:56:26.689 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-01T03:56:26.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T03:56:26.717 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-01T03:56:26.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-08-01T03:56:26.790 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-08-01T03:56:26.817 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-08-01T03:56:26.842 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-01T03:56:26.852 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-01T03:56:26.864 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-01T03:56:26.865 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi160.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-01T04:02:45.662 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-08-01T04:02:45.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-08-01T04:02:45.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T04:02:45.776 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-08-01T04:02:45.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-08-01T04:02:45.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-08-01T04:02:45.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T04:02:46.016 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-08-01T04:02:46.094 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-08-01T04:02:46.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-08-01T04:02:46.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-01T04:02:46.171 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-08-01T04:02:46.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-08-01T04:02:46.253 INFO:teuthology.run_tasks:Running task clock... 2024-08-01T04:02:46.263 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-01T04:02:46.263 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-01T04:02:46.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-01T04:02:46.271 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-01T04:02:46.271 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-01T04:02:46.274 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-01T04:02:46.274 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-01T04:02:46.301 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-01T04:02:46.315 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-01T04:02:46.326 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-01T04:02:46.337 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-08-01T04:02:46.340 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-01T04:02:46.348 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-08-01T04:02:46.361 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-01T04:02:46.363 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-01T04:02:46.365 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-08-01T04:02:46.374 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-01T04:02:46.376 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-08-01T04:02:46.390 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-01T04:02:46.404 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-01T04:02:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-01T04:02:46.438 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-08-01T04:02:46.441 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-01T04:02:46.441 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-08-01T04:02:46.441 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.441 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.441 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.441 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.471 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-08-01T04:02:46.474 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-01T04:02:46.474 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-08-01T04:02:46.474 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.475 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.482 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-08-01T04:02:46.515 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-08-01T04:02:46.559 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-01T04:02:46.604 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-01T04:02:46.754 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-08-01T04:02:46.762 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-01T04:02:46.762 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-08-01T04:02:46.762 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.762 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.763 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.763 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:02:46.764 INFO:teuthology.run_tasks:Running task install... 2024-08-01T04:02:46.773 DEBUG:teuthology.task.install:project ceph 2024-08-01T04:02:46.773 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776'}} 2024-08-01T04:02:46.773 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776'} 2024-08-01T04:02:46.773 INFO:teuthology.task.install:Using flavor: default 2024-08-01T04:02:46.779 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-01T04:02:46.779 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-01T04:02:46.780 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776', 'tag': None, 'wait_for_package': False} 2024-08-01T04:02:46.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:02:46.782 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776', 'tag': None, 'wait_for_package': False} 2024-08-01T04:02:46.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:02:46.783 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776', 'tag': None, 'wait_for_package': False} 2024-08-01T04:02:46.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:02:46.935 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/ 2024-08-01T04:02:46.935 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1223.gf9911e17 2024-08-01T04:02:46.938 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/ 2024-08-01T04:02:46.938 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1223.gf9911e17 2024-08-01T04:02:46.945 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/ 2024-08-01T04:02:46.945 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1223.gf9911e17 2024-08-01T04:02:47.044 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-01T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-01T04:02:47.051 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-01T04:02:47.052 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:02:47.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-01T04:02:47.056 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-01T04:02:47.057 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:02:47.057 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-01T04:02:47.079 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-01T04:02:47.080 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f9911e17b12222fbe598fff73cd3ad10d20c5776/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-01T04:02:47.082 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-01T04:02:47.082 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f9911e17b12222fbe598fff73cd3ad10d20c5776/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-01T04:02:47.137 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-01T04:02:47.137 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f9911e17b12222fbe598fff73cd3ad10d20c5776/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-01T04:02:47.146 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-01T04:02:47.147 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-01T04:02:47.220 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-01T04:02:47.221 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-01T04:02:47.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-01T04:02:47.283 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-08-01T04:02:47.284 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-08-01T04:02:47.286 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-08-01T04:02:47.287 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-08-01T04:02:47.390 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-01T04:02:47.470 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-08-01T04:02:47.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-08-01T04:02:47.509 INFO:teuthology.orchestra.run.smithi160.stdout:45 files removed 2024-08-01T04:02:47.514 INFO:teuthology.orchestra.run.smithi163.stdout:45 files removed 2024-08-01T04:02:47.536 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-01T04:02:47.541 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-01T04:02:47.796 INFO:teuthology.orchestra.run.smithi003.stdout:45 files removed 2024-08-01T04:02:47.835 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-01T04:02:48.113 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 269 kB/s | 84 kB 00:00 2024-08-01T04:02:48.115 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 262 kB/s | 84 kB 00:00 2024-08-01T04:02:48.431 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-08-01T04:02:48.433 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-08-01T04:02:48.658 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 229 kB/s | 84 kB 00:00 2024-08-01T04:02:48.739 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-08-01T04:02:48.759 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-08-01T04:02:49.117 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-08-01T04:02:49.567 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-08-01T04:02:50.199 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-08-01T04:02:51.309 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.2 MB 00:02 2024-08-01T04:02:52.221 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-08-01T04:02:54.621 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 20 MB 00:02 2024-08-01T04:02:55.073 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 1.3 MB/s | 8.2 MB 00:06 2024-08-01T04:02:58.001 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-08-01T04:02:59.084 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.6 MB 00:02 2024-08-01T04:02:59.185 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.6 MB 00:00 2024-08-01T04:03:00.754 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 18 kB 00:00 2024-08-01T04:03:01.694 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 18 kB 00:01 2024-08-01T04:03:02.307 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 16 MB/s | 22 MB 00:01 2024-08-01T04:03:02.624 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-08-01T04:03:05.349 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.6 MB 00:03 2024-08-01T04:03:07.449 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 18 kB 00:00 2024-08-01T04:03:07.979 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-08-01T04:03:08.512 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-08-01T04:03:08.903 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-08-01T04:03:10.256 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-01T04:03:10.256 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-01T04:03:10.268 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-01T04:03:10.268 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-01T04:03:10.269 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-01T04:03:10.270 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-01T04:03:10.272 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-01T04:03:10.274 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-01T04:03:10.275 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-01T04:03:10.339 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 6.6 k 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 5.4 M 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 896 k 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 145 k 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 1.5 M 2024-08-01T04:03:10.360 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 146 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 3.6 M 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 7.4 M 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 49 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 11 M 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 49 M 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 274 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 765 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 225 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 27 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 18 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-01T04:03:10.361 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 32 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 751 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 127 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 188 k 2024-08-01T04:03:10.362 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 369 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 350 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 117 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 85 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.1 M 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 170 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-01T04:03:10.363 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.4 M 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.2 M 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-01T04:03:10.364 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 21 M 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 32 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 2.4 M 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 249 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 4.7 M 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 17 M 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 16 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 25 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-01T04:03:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 163 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 507 k 2024-08-01T04:03:10.366 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 5.4 M 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-01T04:03:10.367 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 45 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 136 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-01T04:03:10.368 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-01T04:03:10.369 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-01T04:03:10.370 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-01T04:03:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-01T04:03:10.372 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout:Install 148 Packages 2024-08-01T04:03:10.373 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 12 Packages 2024-08-01T04:03:10.374 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:03:10.374 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 228 M 2024-08-01T04:03:10.374 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-08-01T04:03:10.905 INFO:teuthology.orchestra.run.smithi160.stdout:(1/160): ceph-19.1.0-1223.gf9911e17.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-08-01T04:03:11.164 INFO:teuthology.orchestra.run.smithi160.stdout:(2/160): ceph-fuse-19.1.0-1223.gf9911e17.el9.x8 3.4 MB/s | 896 kB 00:00 2024-08-01T04:03:11.248 INFO:teuthology.orchestra.run.smithi160.stdout:(3/160): ceph-immutable-object-cache-19.1.0-122 1.7 MB/s | 145 kB 00:00 2024-08-01T04:03:11.323 INFO:teuthology.orchestra.run.smithi160.stdout:(4/160): ceph-base-19.1.0-1223.gf9911e17.el9.x8 9.3 MB/s | 5.4 MB 00:00 2024-08-01T04:03:11.448 INFO:teuthology.orchestra.run.smithi160.stdout:(5/160): ceph-mgr-19.1.0-1223.gf9911e17.el9.x86 12 MB/s | 1.5 MB 00:00 2024-08-01T04:03:11.523 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-01T04:03:11.524 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-01T04:03:11.534 INFO:teuthology.orchestra.run.smithi160.stdout:(6/160): ceph-mds-19.1.0-1223.gf9911e17.el9.x86 8.4 MB/s | 2.4 MB 00:00 2024-08-01T04:03:11.546 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-01T04:03:11.547 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-01T04:03:11.549 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-01T04:03:11.550 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-01T04:03:11.552 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-01T04:03:11.556 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-01T04:03:11.557 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-01T04:03:11.636 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 6.6 k 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 5.4 M 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 896 k 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 145 k 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 1.5 M 2024-08-01T04:03:11.659 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 146 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 3.6 M 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 7.4 M 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 49 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 11 M 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 49 M 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 274 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 765 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 225 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 27 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 18 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-01T04:03:11.660 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 32 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 751 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 127 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 188 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 369 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 350 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 117 k 2024-08-01T04:03:11.661 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 85 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.1 M 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 170 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-01T04:03:11.662 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.4 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.2 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 21 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 32 k 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 2.4 M 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 249 k 2024-08-01T04:03:11.663 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 4.7 M 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 17 M 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 16 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 25 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 163 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-01T04:03:11.664 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 507 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 5.4 M 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-01T04:03:11.665 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-01T04:03:11.666 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 45 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 136 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-01T04:03:11.667 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-01T04:03:11.668 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-01T04:03:11.669 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-01T04:03:11.670 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-08-01T04:03:11.671 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout:Install 148 Packages 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 12 Packages 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 228 M 2024-08-01T04:03:11.672 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-08-01T04:03:11.865 INFO:teuthology.orchestra.run.smithi160.stdout:(7/160): ceph-common-19.1.0-1223.gf9911e17.el9. 19 MB/s | 21 MB 00:01 2024-08-01T04:03:11.941 INFO:teuthology.orchestra.run.smithi160.stdout:(8/160): ceph-mon-19.1.0-1223.gf9911e17.el9.x86 9.6 MB/s | 4.7 MB 00:00 2024-08-01T04:03:11.983 INFO:teuthology.orchestra.run.smithi160.stdout:(9/160): ceph-selinux-19.1.0-1223.gf9911e17.el9 605 kB/s | 25 kB 00:00 2024-08-01T04:03:12.658 INFO:teuthology.orchestra.run.smithi160.stdout:(10/160): ceph-osd-19.1.0-1223.gf9911e17.el9.x8 15 MB/s | 17 MB 00:01 2024-08-01T04:03:12.708 INFO:teuthology.orchestra.run.smithi160.stdout:(11/160): cephfs-mirror-19.1.0-1223.gf9911e17.e 4.4 MB/s | 225 kB 00:00 2024-08-01T04:03:12.783 INFO:teuthology.orchestra.run.smithi160.stdout:(12/160): cephfs-shell-19.1.0-1223.gf9911e17.el 362 kB/s | 27 kB 00:00 2024-08-01T04:03:12.858 INFO:teuthology.orchestra.run.smithi160.stdout:(13/160): libcephfs-devel-19.1.0-1223.gf9911e17 424 kB/s | 32 kB 00:00 2024-08-01T04:03:13.009 INFO:teuthology.orchestra.run.smithi160.stdout:(14/160): ceph-radosgw-19.1.0-1223.gf9911e17.el 9.2 MB/s | 11 MB 00:01 2024-08-01T04:03:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:(15/160): libcephfs2-19.1.0-1223.gf9911e17.el9. 4.0 MB/s | 751 kB 00:00 2024-08-01T04:03:13.046 INFO:teuthology.orchestra.run.smithi003.stdout:(1/160): ceph-19.1.0-1223.gf9911e17.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-08-01T04:03:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:(16/160): libcephsqlite-19.1.0-1223.gf9911e17.e 2.7 MB/s | 163 kB 00:00 2024-08-01T04:03:13.093 INFO:teuthology.orchestra.run.smithi160.stdout:(17/160): librados-devel-19.1.0-1223.gf9911e17. 2.5 MB/s | 127 kB 00:00 2024-08-01T04:03:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:(18/160): libradosstriper1-19.1.0-1223.gf9911e1 8.5 MB/s | 507 kB 00:00 2024-08-01T04:03:13.210 INFO:teuthology.orchestra.run.smithi160.stdout:(19/160): python3-ceph-argparse-19.1.0-1223.gf9 544 kB/s | 45 kB 00:00 2024-08-01T04:03:13.469 INFO:teuthology.orchestra.run.smithi160.stdout:(20/160): librgw2-19.1.0-1223.gf9911e17.el9.x86 14 MB/s | 5.4 MB 00:00 2024-08-01T04:03:13.546 INFO:teuthology.orchestra.run.smithi003.stdout:(2/160): ceph-fuse-19.1.0-1223.gf9911e17.el9.x8 1.8 MB/s | 896 kB 00:00 2024-08-01T04:03:13.552 INFO:teuthology.orchestra.run.smithi160.stdout:(21/160): python3-cephfs-19.1.0-1223.gf9911e17. 2.2 MB/s | 188 kB 00:00 2024-08-01T04:03:13.644 INFO:teuthology.orchestra.run.smithi160.stdout:(22/160): python3-rados-19.1.0-1223.gf9911e17.e 3.9 MB/s | 369 kB 00:00 2024-08-01T04:03:13.688 INFO:teuthology.orchestra.run.smithi003.stdout:(3/160): ceph-base-19.1.0-1223.gf9911e17.el9.x8 5.4 MB/s | 5.4 MB 00:01 2024-08-01T04:03:13.705 INFO:teuthology.orchestra.run.smithi003.stdout:(4/160): ceph-immutable-object-cache-19.1.0-122 920 kB/s | 145 kB 00:00 2024-08-01T04:03:13.776 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-08-01T04:03:13.855 INFO:teuthology.orchestra.run.smithi003.stdout:(5/160): ceph-mds-19.1.0-1223.gf9911e17.el9.x86 14 MB/s | 2.4 MB 00:00 2024-08-01T04:03:14.047 INFO:teuthology.orchestra.run.smithi003.stdout:(6/160): ceph-mgr-19.1.0-1223.gf9911e17.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-08-01T04:03:14.078 INFO:teuthology.orchestra.run.smithi160.stdout:(23/160): ceph-test-19.1.0-1223.gf9911e17.el9.x 23 MB/s | 49 MB 00:02 2024-08-01T04:03:14.103 INFO:teuthology.orchestra.run.smithi160.stdout:(24/160): python3-ceph-common-19.1.0-1223.gf991 152 kB/s | 136 kB 00:00 2024-08-01T04:03:14.129 INFO:teuthology.orchestra.run.smithi160.stdout:(25/160): python3-rgw-19.1.0-1223.gf9911e17.el9 2.3 MB/s | 117 kB 00:00 2024-08-01T04:03:14.154 INFO:teuthology.orchestra.run.smithi160.stdout:(26/160): python3-rbd-19.1.0-1223.gf9911e17.el9 687 kB/s | 350 kB 00:00 2024-08-01T04:03:14.172 INFO:teuthology.orchestra.run.smithi003.stdout:(7/160): ceph-mon-19.1.0-1223.gf9911e17.el9.x86 15 MB/s | 4.7 MB 00:00 2024-08-01T04:03:14.179 INFO:teuthology.orchestra.run.smithi160.stdout:(27/160): rbd-fuse-19.1.0-1223.gf9911e17.el9.x8 1.1 MB/s | 85 kB 00:00 2024-08-01T04:03:14.271 INFO:teuthology.orchestra.run.smithi160.stdout:(28/160): rbd-nbd-19.1.0-1223.gf9911e17.el9.x86 1.8 MB/s | 170 kB 00:00 2024-08-01T04:03:14.288 INFO:teuthology.orchestra.run.smithi160.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1223.g 293 kB/s | 32 kB 00:00 2024-08-01T04:03:14.338 INFO:teuthology.orchestra.run.smithi160.stdout:(30/160): rbd-mirror-19.1.0-1223.gf9911e17.el9. 15 MB/s | 3.1 MB 00:00 2024-08-01T04:03:14.364 INFO:teuthology.orchestra.run.smithi160.stdout:(31/160): ceph-mgr-cephadm-19.1.0-1223.gf9911e1 1.5 MB/s | 146 kB 00:00 2024-08-01T04:03:14.455 INFO:teuthology.orchestra.run.smithi160.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1223.gf9 2.7 MB/s | 249 kB 00:00 2024-08-01T04:03:14.531 INFO:teuthology.orchestra.run.smithi160.stdout:(33/160): ceph-mgr-rook-19.1.0-1223.gf9911e17.e 658 kB/s | 49 kB 00:00 2024-08-01T04:03:14.648 INFO:teuthology.orchestra.run.smithi003.stdout:(8/160): ceph-common-19.1.0-1223.gf9911e17.el9. 11 MB/s | 21 MB 00:01 2024-08-01T04:03:14.656 INFO:teuthology.orchestra.run.smithi160.stdout:(34/160): ceph-mgr-diskprediction-local-19.1.0- 23 MB/s | 7.4 MB 00:00 2024-08-01T04:03:14.698 INFO:teuthology.orchestra.run.smithi003.stdout:(9/160): ceph-selinux-19.1.0-1223.gf9911e17.el9 516 kB/s | 25 kB 00:00 2024-08-01T04:03:14.698 INFO:teuthology.orchestra.run.smithi160.stdout:(35/160): ceph-prometheus-alerts-19.1.0-1223.gf 98 kB/s | 16 kB 00:00 2024-08-01T04:03:14.723 INFO:teuthology.orchestra.run.smithi160.stdout:(36/160): ceph-volume-19.1.0-1223.gf9911e17.el9 4.0 MB/s | 274 kB 00:00 2024-08-01T04:03:14.790 INFO:teuthology.orchestra.run.smithi160.stdout:(37/160): ceph-mgr-dashboard-19.1.0-1223.gf9911 7.1 MB/s | 3.6 MB 00:00 2024-08-01T04:03:14.810 INFO:teuthology.orchestra.run.smithi160.stdout:(38/160): cephfs-top-19.1.0-1223.gf9911e17.el9. 218 kB/s | 18 kB 00:00 2024-08-01T04:03:14.841 INFO:teuthology.orchestra.run.smithi160.stdout:(39/160): cephadm-19.1.0-1223.gf9911e17.el9.noa 5.2 MB/s | 765 kB 00:00 2024-08-01T04:03:14.990 INFO:teuthology.orchestra.run.smithi003.stdout:(10/160): ceph-radosgw-19.1.0-1223.gf9911e17.el 13 MB/s | 11 MB 00:00 2024-08-01T04:03:15.048 INFO:teuthology.orchestra.run.smithi003.stdout:(11/160): cephfs-mirror-19.1.0-1223.gf9911e17.e 3.8 MB/s | 225 kB 00:00 2024-08-01T04:03:15.099 INFO:teuthology.orchestra.run.smithi003.stdout:(12/160): cephfs-shell-19.1.0-1223.gf9911e17.el 549 kB/s | 27 kB 00:00 2024-08-01T04:03:15.149 INFO:teuthology.orchestra.run.smithi003.stdout:(13/160): libcephfs-devel-19.1.0-1223.gf9911e17 643 kB/s | 32 kB 00:00 2024-08-01T04:03:15.233 INFO:teuthology.orchestra.run.smithi160.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 180 kB/s | 80 kB 00:00 2024-08-01T04:03:15.249 INFO:teuthology.orchestra.run.smithi003.stdout:(14/160): libcephfs2-19.1.0-1223.gf9911e17.el9. 7.4 MB/s | 751 kB 00:00 2024-08-01T04:03:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:(15/160): libcephsqlite-19.1.0-1223.gf9911e17.e 2.1 MB/s | 163 kB 00:00 2024-08-01T04:03:15.358 INFO:teuthology.orchestra.run.smithi160.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 378 kB/s | 195 kB 00:00 2024-08-01T04:03:15.383 INFO:teuthology.orchestra.run.smithi003.stdout:(16/160): librados-devel-19.1.0-1223.gf9911e17. 2.2 MB/s | 127 kB 00:00 2024-08-01T04:03:15.450 INFO:teuthology.orchestra.run.smithi003.stdout:(17/160): libradosstriper1-19.1.0-1223.gf9911e1 7.5 MB/s | 507 kB 00:00 2024-08-01T04:03:15.475 INFO:teuthology.orchestra.run.smithi160.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-08-01T04:03:15.575 INFO:teuthology.orchestra.run.smithi160.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 740 kB/s | 253 kB 00:00 2024-08-01T04:03:15.600 INFO:teuthology.orchestra.run.smithi160.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-08-01T04:03:15.684 INFO:teuthology.orchestra.run.smithi160.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-08-01T04:03:15.709 INFO:teuthology.orchestra.run.smithi160.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-08-01T04:03:15.776 INFO:teuthology.orchestra.run.smithi003.stdout:(18/160): librgw2-19.1.0-1223.gf9911e17.el9.x86 17 MB/s | 5.4 MB 00:00 2024-08-01T04:03:15.793 INFO:teuthology.orchestra.run.smithi160.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-01T04:03:15.818 INFO:teuthology.orchestra.run.smithi003.stdout:(19/160): python3-ceph-argparse-19.1.0-1223.gf9 1.1 MB/s | 45 kB 00:00 2024-08-01T04:03:15.826 INFO:teuthology.orchestra.run.smithi160.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-08-01T04:03:15.868 INFO:teuthology.orchestra.run.smithi003.stdout:(20/160): python3-ceph-common-19.1.0-1223.gf991 2.7 MB/s | 136 kB 00:00 2024-08-01T04:03:15.927 INFO:teuthology.orchestra.run.smithi003.stdout:(21/160): python3-cephfs-19.1.0-1223.gf9911e17. 3.2 MB/s | 188 kB 00:00 2024-08-01T04:03:15.993 INFO:teuthology.orchestra.run.smithi003.stdout:(22/160): python3-rados-19.1.0-1223.gf9911e17.e 5.5 MB/s | 369 kB 00:00 2024-08-01T04:03:16.002 INFO:teuthology.orchestra.run.smithi160.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-08-01T04:03:16.027 INFO:teuthology.orchestra.run.smithi160.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 534 kB/s | 107 kB 00:00 2024-08-01T04:03:16.052 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-01T04:03:16.052 INFO:teuthology.orchestra.run.smithi160.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-08-01T04:03:16.053 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-01T04:03:16.060 INFO:teuthology.orchestra.run.smithi003.stdout:(23/160): python3-rbd-19.1.0-1223.gf9911e17.el9 5.2 MB/s | 350 kB 00:00 2024-08-01T04:03:16.063 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-01T04:03:16.064 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-01T04:03:16.065 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-01T04:03:16.066 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-01T04:03:16.067 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-01T04:03:16.070 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-01T04:03:16.070 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-01T04:03:16.103 INFO:teuthology.orchestra.run.smithi160.stdout:(52/160): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-08-01T04:03:16.119 INFO:teuthology.orchestra.run.smithi003.stdout:(24/160): python3-rgw-19.1.0-1223.gf9911e17.el9 2.0 MB/s | 117 kB 00:00 2024-08-01T04:03:16.135 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-08-01T04:03:16.144 INFO:teuthology.orchestra.run.smithi160.stdout:(53/160): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-08-01T04:03:16.156 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-08-01T04:03:16.156 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-08-01T04:03:16.156 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-08-01T04:03:16.156 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-08-01T04:03:16.156 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 6.6 k 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 5.4 M 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 896 k 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 145 k 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 1.5 M 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 146 k 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 3.6 M 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 7.4 M 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 49 k 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 11 M 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 49 M 2024-08-01T04:03:16.157 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 274 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 765 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 225 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 27 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 18 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 32 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 751 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 127 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-01T04:03:16.158 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 188 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 369 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 350 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 117 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 85 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.1 M 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 170 k 2024-08-01T04:03:16.159 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-01T04:03:16.160 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.4 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 3.2 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 21 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 32 k 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 2.4 M 2024-08-01T04:03:16.161 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 249 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 4.7 M 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 17 M 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1223.gf9911e17.el9 ceph-noarch 16 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 25 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-01T04:03:16.162 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 163 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 507 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 5.4 M 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-01T04:03:16.163 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-01T04:03:16.164 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 45 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:19.1.0-1223.gf9911e17.el9 ceph 136 k 2024-08-01T04:03:16.165 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-01T04:03:16.166 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-01T04:03:16.167 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-01T04:03:16.168 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-01T04:03:16.169 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-01T04:03:16.170 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout:======================================================================================= 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout:Install 148 Packages 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 12 Packages 2024-08-01T04:03:16.171 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:03:16.172 INFO:teuthology.orchestra.run.smithi160.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-08-01T04:03:16.172 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 228 M 2024-08-01T04:03:16.172 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-08-01T04:03:16.178 INFO:teuthology.orchestra.run.smithi003.stdout:(25/160): rbd-fuse-19.1.0-1223.gf9911e17.el9.x8 1.4 MB/s | 85 kB 00:00 2024-08-01T04:03:16.195 INFO:teuthology.orchestra.run.smithi160.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-08-01T04:03:16.220 INFO:teuthology.orchestra.run.smithi160.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-08-01T04:03:16.238 INFO:teuthology.orchestra.run.smithi160.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-08-01T04:03:16.279 INFO:teuthology.orchestra.run.smithi160.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-08-01T04:03:16.338 INFO:teuthology.orchestra.run.smithi160.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-08-01T04:03:16.388 INFO:teuthology.orchestra.run.smithi160.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-08-01T04:03:16.395 INFO:teuthology.orchestra.run.smithi003.stdout:(26/160): rbd-mirror-19.1.0-1223.gf9911e17.el9. 14 MB/s | 3.1 MB 00:00 2024-08-01T04:03:16.414 INFO:teuthology.orchestra.run.smithi160.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-08-01T04:03:16.439 INFO:teuthology.orchestra.run.smithi160.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-08-01T04:03:16.453 INFO:teuthology.orchestra.run.smithi003.stdout:(27/160): rbd-nbd-19.1.0-1223.gf9911e17.el9.x86 2.9 MB/s | 170 kB 00:00 2024-08-01T04:03:16.473 INFO:teuthology.orchestra.run.smithi160.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-08-01T04:03:16.504 INFO:teuthology.orchestra.run.smithi003.stdout:(28/160): ceph-grafana-dashboards-19.1.0-1223.g 638 kB/s | 32 kB 00:00 2024-08-01T04:03:16.523 INFO:teuthology.orchestra.run.smithi160.stdout:(64/160): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-08-01T04:03:16.548 INFO:teuthology.orchestra.run.smithi160.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-08-01T04:03:16.562 INFO:teuthology.orchestra.run.smithi003.stdout:(29/160): ceph-mgr-cephadm-19.1.0-1223.gf9911e1 2.5 MB/s | 146 kB 00:00 2024-08-01T04:03:16.788 INFO:teuthology.orchestra.run.smithi003.stdout:(30/160): ceph-mgr-dashboard-19.1.0-1223.gf9911 16 MB/s | 3.6 MB 00:00 2024-08-01T04:03:16.810 INFO:teuthology.orchestra.run.smithi163.stdout:(1/160): ceph-19.1.0-1223.gf9911e17.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-08-01T04:03:17.052 INFO:teuthology.orchestra.run.smithi163.stdout:(2/160): ceph-fuse-19.1.0-1223.gf9911e17.el9.x8 3.6 MB/s | 896 kB 00:00 2024-08-01T04:03:17.136 INFO:teuthology.orchestra.run.smithi163.stdout:(3/160): ceph-base-19.1.0-1223.gf9911e17.el9.x8 10 MB/s | 5.4 MB 00:00 2024-08-01T04:03:17.161 INFO:teuthology.orchestra.run.smithi163.stdout:(4/160): ceph-immutable-object-cache-19.1.0-122 1.3 MB/s | 145 kB 00:00 2024-08-01T04:03:17.286 INFO:teuthology.orchestra.run.smithi163.stdout:(5/160): ceph-mds-19.1.0-1223.gf9911e17.el9.x86 16 MB/s | 2.4 MB 00:00 2024-08-01T04:03:17.349 INFO:teuthology.orchestra.run.smithi160.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-08-01T04:03:17.355 INFO:teuthology.orchestra.run.smithi003.stdout:(31/160): ceph-test-19.1.0-1223.gf9911e17.el9.x 18 MB/s | 49 MB 00:02 2024-08-01T04:03:17.420 INFO:teuthology.orchestra.run.smithi163.stdout:(6/160): ceph-mgr-19.1.0-1223.gf9911e17.el9.x86 5.9 MB/s | 1.5 MB 00:00 2024-08-01T04:03:17.530 INFO:teuthology.orchestra.run.smithi003.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1223.gf9 1.4 MB/s | 249 kB 00:00 2024-08-01T04:03:17.545 INFO:teuthology.orchestra.run.smithi163.stdout:(7/160): ceph-mon-19.1.0-1223.gf9911e17.el9.x86 18 MB/s | 4.7 MB 00:00 2024-08-01T04:03:17.605 INFO:teuthology.orchestra.run.smithi003.stdout:(33/160): ceph-mgr-rook-19.1.0-1223.gf9911e17.e 664 kB/s | 49 kB 00:00 2024-08-01T04:03:17.714 INFO:teuthology.orchestra.run.smithi003.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1223.gf 153 kB/s | 16 kB 00:00 2024-08-01T04:03:17.814 INFO:teuthology.orchestra.run.smithi003.stdout:(35/160): ceph-volume-19.1.0-1223.gf9911e17.el9 2.7 MB/s | 274 kB 00:00 2024-08-01T04:03:18.012 INFO:teuthology.orchestra.run.smithi163.stdout:(8/160): ceph-common-19.1.0-1223.gf9911e17.el9. 15 MB/s | 21 MB 00:01 2024-08-01T04:03:18.056 INFO:teuthology.orchestra.run.smithi003.stdout:(36/160): ceph-osd-19.1.0-1223.gf9911e17.el9.x8 4.2 MB/s | 17 MB 00:04 2024-08-01T04:03:18.062 INFO:teuthology.orchestra.run.smithi163.stdout:(9/160): ceph-selinux-19.1.0-1223.gf9911e17.el9 506 kB/s | 25 kB 00:00 2024-08-01T04:03:18.124 INFO:teuthology.orchestra.run.smithi160.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-08-01T04:03:18.198 INFO:teuthology.orchestra.run.smithi003.stdout:(37/160): ceph-mgr-diskprediction-local-19.1.0- 5.3 MB/s | 7.4 MB 00:01 2024-08-01T04:03:18.223 INFO:teuthology.orchestra.run.smithi003.stdout:(38/160): cephfs-top-19.1.0-1223.gf9911e17.el9. 110 kB/s | 18 kB 00:00 2024-08-01T04:03:18.238 INFO:teuthology.orchestra.run.smithi163.stdout:(10/160): ceph-radosgw-19.1.0-1223.gf9911e17.el 15 MB/s | 11 MB 00:00 2024-08-01T04:03:18.274 INFO:teuthology.orchestra.run.smithi003.stdout:(39/160): cephadm-19.1.0-1223.gf9911e17.el9.noa 1.6 MB/s | 765 kB 00:00 2024-08-01T04:03:18.313 INFO:teuthology.orchestra.run.smithi163.stdout:(11/160): cephfs-mirror-19.1.0-1223.gf9911e17.e 2.9 MB/s | 225 kB 00:00 2024-08-01T04:03:18.357 INFO:teuthology.orchestra.run.smithi160.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-08-01T04:03:18.364 INFO:teuthology.orchestra.run.smithi163.stdout:(12/160): cephfs-shell-19.1.0-1223.gf9911e17.el 543 kB/s | 27 kB 00:00 2024-08-01T04:03:18.390 INFO:teuthology.orchestra.run.smithi003.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 418 kB/s | 80 kB 00:00 2024-08-01T04:03:18.414 INFO:teuthology.orchestra.run.smithi163.stdout:(13/160): libcephfs-devel-19.1.0-1223.gf9911e17 638 kB/s | 32 kB 00:00 2024-08-01T04:03:18.458 INFO:teuthology.orchestra.run.smithi003.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-08-01T04:03:18.474 INFO:teuthology.orchestra.run.smithi160.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-08-01T04:03:18.490 INFO:teuthology.orchestra.run.smithi163.stdout:(14/160): libcephfs2-19.1.0-1223.gf9911e17.el9. 9.8 MB/s | 751 kB 00:00 2024-08-01T04:03:18.491 INFO:teuthology.orchestra.run.smithi003.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-08-01T04:03:18.541 INFO:teuthology.orchestra.run.smithi003.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-08-01T04:03:18.566 INFO:teuthology.orchestra.run.smithi003.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-08-01T04:03:18.609 INFO:teuthology.orchestra.run.smithi003.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-08-01T04:03:18.642 INFO:teuthology.orchestra.run.smithi003.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-08-01T04:03:18.656 INFO:teuthology.orchestra.run.smithi163.stdout:(15/160): ceph-osd-19.1.0-1223.gf9911e17.el9.x8 14 MB/s | 17 MB 00:01 2024-08-01T04:03:18.658 INFO:teuthology.orchestra.run.smithi160.stdout:(70/160): openblas-openmp-0.3.21-2.el9.x86_64.r 2.2 MB/s | 4.7 MB 00:02 2024-08-01T04:03:18.681 INFO:teuthology.orchestra.run.smithi163.stdout:(16/160): libcephsqlite-19.1.0-1223.gf9911e17.e 852 kB/s | 163 kB 00:00 2024-08-01T04:03:18.691 INFO:teuthology.orchestra.run.smithi160.stdout:(71/160): postgresql-contrib-13.14-1.el9.x86_64 1.6 MB/s | 911 kB 00:00 2024-08-01T04:03:18.692 INFO:teuthology.orchestra.run.smithi003.stdout:(47/160): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-08-01T04:03:18.707 INFO:teuthology.orchestra.run.smithi163.stdout:(17/160): librados-devel-19.1.0-1223.gf9911e17. 2.5 MB/s | 127 kB 00:00 2024-08-01T04:03:18.718 INFO:teuthology.orchestra.run.smithi003.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-01T04:03:18.740 INFO:teuthology.orchestra.run.smithi163.stdout:(18/160): libradosstriper1-19.1.0-1223.gf9911e1 8.4 MB/s | 507 kB 00:00 2024-08-01T04:03:18.775 INFO:teuthology.orchestra.run.smithi160.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-08-01T04:03:18.799 INFO:teuthology.orchestra.run.smithi163.stdout:(19/160): python3-ceph-argparse-19.1.0-1223.gf9 774 kB/s | 45 kB 00:00 2024-08-01T04:03:18.866 INFO:teuthology.orchestra.run.smithi163.stdout:(20/160): python3-ceph-common-19.1.0-1223.gf991 2.0 MB/s | 136 kB 00:00 2024-08-01T04:03:18.950 INFO:teuthology.orchestra.run.smithi163.stdout:(21/160): librgw2-19.1.0-1223.gf9911e17.el9.x86 22 MB/s | 5.4 MB 00:00 2024-08-01T04:03:18.974 INFO:teuthology.orchestra.run.smithi163.stdout:(22/160): python3-cephfs-19.1.0-1223.gf9911e17. 1.7 MB/s | 188 kB 00:00 2024-08-01T04:03:19.008 INFO:teuthology.orchestra.run.smithi163.stdout:(23/160): python3-rados-19.1.0-1223.gf9911e17.e 6.2 MB/s | 369 kB 00:00 2024-08-01T04:03:19.033 INFO:teuthology.orchestra.run.smithi163.stdout:(24/160): python3-rbd-19.1.0-1223.gf9911e17.el9 5.8 MB/s | 350 kB 00:00 2024-08-01T04:03:19.059 INFO:teuthology.orchestra.run.smithi163.stdout:(25/160): python3-rgw-19.1.0-1223.gf9911e17.el9 2.3 MB/s | 117 kB 00:00 2024-08-01T04:03:19.084 INFO:teuthology.orchestra.run.smithi163.stdout:(26/160): rbd-fuse-19.1.0-1223.gf9911e17.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-01T04:03:19.159 INFO:teuthology.orchestra.run.smithi163.stdout:(27/160): rbd-nbd-19.1.0-1223.gf9911e17.el9.x86 2.2 MB/s | 170 kB 00:00 2024-08-01T04:03:19.175 INFO:teuthology.orchestra.run.smithi160.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-08-01T04:03:19.218 INFO:teuthology.orchestra.run.smithi163.stdout:(28/160): rbd-mirror-19.1.0-1223.gf9911e17.el9. 20 MB/s | 3.1 MB 00:00 2024-08-01T04:03:19.225 INFO:teuthology.orchestra.run.smithi160.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-01T04:03:19.243 INFO:teuthology.orchestra.run.smithi163.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1223.g 382 kB/s | 32 kB 00:00 2024-08-01T04:03:19.269 INFO:teuthology.orchestra.run.smithi163.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1223.gf9911e1 2.8 MB/s | 146 kB 00:00 2024-08-01T04:03:19.310 INFO:teuthology.orchestra.run.smithi003.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 181 kB/s | 107 kB 00:00 2024-08-01T04:03:19.368 INFO:teuthology.orchestra.run.smithi160.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-08-01T04:03:19.443 INFO:teuthology.orchestra.run.smithi003.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 861 kB/s | 689 kB 00:00 2024-08-01T04:03:19.469 INFO:teuthology.orchestra.run.smithi003.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 139 kB/s | 22 kB 00:00 2024-08-01T04:03:19.518 INFO:teuthology.orchestra.run.smithi160.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-08-01T04:03:19.519 INFO:teuthology.orchestra.run.smithi003.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-08-01T04:03:19.527 INFO:teuthology.orchestra.run.smithi163.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1223.gf9911 13 MB/s | 3.6 MB 00:00 2024-08-01T04:03:19.568 INFO:teuthology.orchestra.run.smithi160.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-08-01T04:03:19.577 INFO:teuthology.orchestra.run.smithi003.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-08-01T04:03:19.603 INFO:teuthology.orchestra.run.smithi003.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 248 kB/s | 33 kB 00:00 2024-08-01T04:03:19.618 INFO:teuthology.orchestra.run.smithi160.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-08-01T04:03:19.628 INFO:teuthology.orchestra.run.smithi163.stdout:(32/160): ceph-mgr-diskprediction-local-19.1.0- 21 MB/s | 7.4 MB 00:00 2024-08-01T04:03:19.653 INFO:teuthology.orchestra.run.smithi163.stdout:(33/160): ceph-mgr-modules-core-19.1.0-1223.gf9 1.9 MB/s | 249 kB 00:00 2024-08-01T04:03:19.678 INFO:teuthology.orchestra.run.smithi163.stdout:(34/160): ceph-mgr-rook-19.1.0-1223.gf9911e17.e 985 kB/s | 49 kB 00:00 2024-08-01T04:03:19.711 INFO:teuthology.orchestra.run.smithi163.stdout:(35/160): ceph-prometheus-alerts-19.1.0-1223.gf 281 kB/s | 16 kB 00:00 2024-08-01T04:03:19.743 INFO:teuthology.orchestra.run.smithi160.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-08-01T04:03:19.754 INFO:teuthology.orchestra.run.smithi163.stdout:(36/160): ceph-volume-19.1.0-1223.gf9911e17.el9 3.5 MB/s | 274 kB 00:00 2024-08-01T04:03:19.787 INFO:teuthology.orchestra.run.smithi163.stdout:(37/160): cephadm-19.1.0-1223.gf9911e17.el9.noa 9.9 MB/s | 765 kB 00:00 2024-08-01T04:03:19.794 INFO:teuthology.orchestra.run.smithi160.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-08-01T04:03:19.812 INFO:teuthology.orchestra.run.smithi163.stdout:(38/160): cephfs-top-19.1.0-1223.gf9911e17.el9. 311 kB/s | 18 kB 00:00 2024-08-01T04:03:19.853 INFO:teuthology.orchestra.run.smithi003.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.0 MB/s | 3.0 MB 00:00 2024-08-01T04:03:19.878 INFO:teuthology.orchestra.run.smithi003.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 61 kB/s | 18 kB 00:00 2024-08-01T04:03:19.903 INFO:teuthology.orchestra.run.smithi003.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 37 kB/s | 11 kB 00:00 2024-08-01T04:03:19.915 INFO:teuthology.orchestra.run.smithi163.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 626 kB/s | 80 kB 00:00 2024-08-01T04:03:19.962 INFO:teuthology.orchestra.run.smithi003.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-08-01T04:03:19.974 INFO:teuthology.orchestra.run.smithi163.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-08-01T04:03:19.999 INFO:teuthology.orchestra.run.smithi163.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-08-01T04:03:20.004 INFO:teuthology.orchestra.run.smithi003.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 456 kB/s | 45 kB 00:00 2024-08-01T04:03:20.024 INFO:teuthology.orchestra.run.smithi163.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-08-01T04:03:20.046 INFO:teuthology.orchestra.run.smithi003.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-08-01T04:03:20.066 INFO:teuthology.orchestra.run.smithi163.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-08-01T04:03:20.096 INFO:teuthology.orchestra.run.smithi003.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 4.3 MB/s | 581 kB 00:00 2024-08-01T04:03:20.121 INFO:teuthology.orchestra.run.smithi003.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 228 kB/s | 25 kB 00:00 2024-08-01T04:03:20.155 INFO:teuthology.orchestra.run.smithi003.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 463 kB/s | 47 kB 00:00 2024-08-01T04:03:20.214 INFO:teuthology.orchestra.run.smithi003.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-08-01T04:03:20.247 INFO:teuthology.orchestra.run.smithi003.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-08-01T04:03:20.273 INFO:teuthology.orchestra.run.smithi003.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 302 kB/s | 35 kB 00:00 2024-08-01T04:03:20.483 INFO:teuthology.orchestra.run.smithi163.stdout:(44/160): ceph-test-19.1.0-1223.gf9911e17.el9.x 20 MB/s | 49 MB 00:02 2024-08-01T04:03:20.489 INFO:teuthology.orchestra.run.smithi003.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-08-01T04:03:20.508 INFO:teuthology.orchestra.run.smithi163.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 306 kB/s | 135 kB 00:00 2024-08-01T04:03:20.542 INFO:teuthology.orchestra.run.smithi163.stdout:(46/160): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-08-01T04:03:20.567 INFO:teuthology.orchestra.run.smithi163.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-08-01T04:03:20.592 INFO:teuthology.orchestra.run.smithi163.stdout:(48/160): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-08-01T04:03:20.673 INFO:teuthology.orchestra.run.smithi003.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-08-01T04:03:20.748 INFO:teuthology.orchestra.run.smithi003.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-08-01T04:03:20.782 INFO:teuthology.orchestra.run.smithi003.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 3.1 MB/s | 911 kB 00:00 2024-08-01T04:03:20.807 INFO:teuthology.orchestra.run.smithi003.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-08-01T04:03:21.082 INFO:teuthology.orchestra.run.smithi003.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-08-01T04:03:21.108 INFO:teuthology.orchestra.run.smithi003.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 292 kB/s | 87 kB 00:00 2024-08-01T04:03:21.109 INFO:teuthology.orchestra.run.smithi163.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 207 kB/s | 107 kB 00:00 2024-08-01T04:03:21.158 INFO:teuthology.orchestra.run.smithi003.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-08-01T04:03:21.285 INFO:teuthology.orchestra.run.smithi163.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 928 kB/s | 689 kB 00:00 2024-08-01T04:03:21.310 INFO:teuthology.orchestra.run.smithi163.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 108 kB/s | 22 kB 00:00 2024-08-01T04:03:21.316 INFO:teuthology.orchestra.run.smithi003.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 300 kB/s | 61 kB 00:00 2024-08-01T04:03:21.335 INFO:teuthology.orchestra.run.smithi163.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-08-01T04:03:21.427 INFO:teuthology.orchestra.run.smithi163.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-08-01T04:03:21.452 INFO:teuthology.orchestra.run.smithi163.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-08-01T04:03:21.467 INFO:teuthology.orchestra.run.smithi003.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-08-01T04:03:21.469 INFO:teuthology.orchestra.run.smithi160.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:02 2024-08-01T04:03:21.492 INFO:teuthology.orchestra.run.smithi003.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 732 kB/s | 244 kB 00:00 2024-08-01T04:03:21.526 INFO:teuthology.orchestra.run.smithi003.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-08-01T04:03:21.594 INFO:teuthology.orchestra.run.smithi160.stdout:(82/160): python3-babel-2.9.1-2.el9.noarch.rpm 2.1 MB/s | 6.0 MB 00:02 2024-08-01T04:03:21.601 INFO:teuthology.orchestra.run.smithi003.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 244 kB/s | 26 kB 00:00 2024-08-01T04:03:21.627 INFO:teuthology.orchestra.run.smithi003.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 147 kB/s | 19 kB 00:00 2024-08-01T04:03:21.636 INFO:teuthology.orchestra.run.smithi163.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-08-01T04:03:21.653 INFO:teuthology.orchestra.run.smithi160.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-08-01T04:03:21.660 INFO:teuthology.orchestra.run.smithi003.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-08-01T04:03:21.661 INFO:teuthology.orchestra.run.smithi163.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 78 kB/s | 18 kB 00:00 2024-08-01T04:03:21.678 INFO:teuthology.orchestra.run.smithi160.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-08-01T04:03:21.686 INFO:teuthology.orchestra.run.smithi163.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 48 kB/s | 11 kB 00:00 2024-08-01T04:03:21.727 INFO:teuthology.orchestra.run.smithi003.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 356 kB/s | 35 kB 00:00 2024-08-01T04:03:21.728 INFO:teuthology.orchestra.run.smithi160.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-08-01T04:03:21.745 INFO:teuthology.orchestra.run.smithi163.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-08-01T04:03:21.754 INFO:teuthology.orchestra.run.smithi160.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 764 kB/s | 77 kB 00:00 2024-08-01T04:03:21.787 INFO:teuthology.orchestra.run.smithi163.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-08-01T04:03:21.820 INFO:teuthology.orchestra.run.smithi163.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-08-01T04:03:21.854 INFO:teuthology.orchestra.run.smithi163.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-08-01T04:03:21.862 INFO:teuthology.orchestra.run.smithi160.stdout:(87/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-08-01T04:03:21.887 INFO:teuthology.orchestra.run.smithi163.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 256 kB/s | 25 kB 00:00 2024-08-01T04:03:21.913 INFO:teuthology.orchestra.run.smithi160.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-08-01T04:03:21.921 INFO:teuthology.orchestra.run.smithi163.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-08-01T04:03:21.935 INFO:teuthology.orchestra.run.smithi003.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-08-01T04:03:21.955 INFO:teuthology.orchestra.run.smithi163.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-08-01T04:03:21.960 INFO:teuthology.orchestra.run.smithi003.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 836 kB/s | 250 kB 00:00 2024-08-01T04:03:21.980 INFO:teuthology.orchestra.run.smithi160.stdout:(89/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.3 MB/s | 5.1 MB 00:02 2024-08-01T04:03:21.985 INFO:teuthology.orchestra.run.smithi003.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 905 kB/s | 222 kB 00:00 2024-08-01T04:03:21.988 INFO:teuthology.orchestra.run.smithi163.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-08-01T04:03:22.005 INFO:teuthology.orchestra.run.smithi160.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-08-01T04:03:22.022 INFO:teuthology.orchestra.run.smithi163.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-08-01T04:03:22.030 INFO:teuthology.orchestra.run.smithi160.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-01T04:03:22.080 INFO:teuthology.orchestra.run.smithi160.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-08-01T04:03:22.094 INFO:teuthology.orchestra.run.smithi003.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 722 kB/s | 77 kB 00:00 2024-08-01T04:03:22.106 INFO:teuthology.orchestra.run.smithi160.stdout:(93/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-08-01T04:03:22.119 INFO:teuthology.orchestra.run.smithi003.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 321 kB/s | 42 kB 00:00 2024-08-01T04:03:22.131 INFO:teuthology.orchestra.run.smithi160.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 876 kB/s | 44 kB 00:00 2024-08-01T04:03:22.153 INFO:teuthology.orchestra.run.smithi003.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-08-01T04:03:22.189 INFO:teuthology.orchestra.run.smithi160.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-08-01T04:03:22.197 INFO:teuthology.orchestra.run.smithi163.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-08-01T04:03:22.229 INFO:teuthology.orchestra.run.smithi003.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-08-01T04:03:22.240 INFO:teuthology.orchestra.run.smithi160.stdout:(96/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-08-01T04:03:22.254 INFO:teuthology.orchestra.run.smithi003.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 279 kB 00:00 2024-08-01T04:03:22.274 INFO:teuthology.orchestra.run.smithi160.stdout:(97/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-08-01T04:03:22.279 INFO:teuthology.orchestra.run.smithi003.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 408 kB/s | 51 kB 00:00 2024-08-01T04:03:22.324 INFO:teuthology.orchestra.run.smithi160.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-01T04:03:22.340 INFO:teuthology.orchestra.run.smithi163.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-08-01T04:03:22.362 INFO:teuthology.orchestra.run.smithi003.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 502 kB/s | 54 kB 00:00 2024-08-01T04:03:22.381 INFO:teuthology.orchestra.run.smithi163.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-08-01T04:03:22.415 INFO:teuthology.orchestra.run.smithi163.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-08-01T04:03:22.416 INFO:teuthology.orchestra.run.smithi160.stdout:(99/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 1.0 MB/s | 96 kB 00:00 2024-08-01T04:03:22.440 INFO:teuthology.orchestra.run.smithi163.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-08-01T04:03:22.441 INFO:teuthology.orchestra.run.smithi160.stdout:(100/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-08-01T04:03:22.471 INFO:teuthology.orchestra.run.smithi003.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-08-01T04:03:22.508 INFO:teuthology.orchestra.run.smithi160.stdout:(101/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-08-01T04:03:22.533 INFO:teuthology.orchestra.run.smithi160.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 543 kB/s | 64 kB 00:00 2024-08-01T04:03:22.580 INFO:teuthology.orchestra.run.smithi003.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 216 kB/s | 44 kB 00:00 2024-08-01T04:03:22.607 INFO:teuthology.orchestra.run.smithi163.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 525 kB/s | 87 kB 00:00 2024-08-01T04:03:22.667 INFO:teuthology.orchestra.run.smithi160.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-08-01T04:03:22.682 INFO:teuthology.orchestra.run.smithi163.stdout:(73/160): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-08-01T04:03:22.716 INFO:teuthology.orchestra.run.smithi163.stdout:(74/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-08-01T04:03:22.734 INFO:teuthology.orchestra.run.smithi160.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-08-01T04:03:22.738 INFO:teuthology.orchestra.run.smithi003.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 247 kB/s | 36 kB 00:00 2024-08-01T04:03:22.759 INFO:teuthology.orchestra.run.smithi160.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-01T04:03:22.776 INFO:teuthology.orchestra.run.smithi160.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-08-01T04:03:22.780 INFO:teuthology.orchestra.run.smithi003.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 747 kB/s | 221 kB 00:00 2024-08-01T04:03:22.900 INFO:teuthology.orchestra.run.smithi163.stdout:(75/160): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-08-01T04:03:22.925 INFO:teuthology.orchestra.run.smithi163.stdout:(76/160): python3-colorama-0.4.6-3.el9.noarch.r 296 kB/s | 61 kB 00:00 2024-08-01T04:03:22.926 INFO:teuthology.orchestra.run.smithi160.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-08-01T04:03:22.943 INFO:teuthology.orchestra.run.smithi160.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-08-01T04:03:22.950 INFO:teuthology.orchestra.run.smithi163.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-08-01T04:03:22.977 INFO:teuthology.orchestra.run.smithi160.stdout:(109/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-08-01T04:03:23.000 INFO:teuthology.orchestra.run.smithi163.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-08-01T04:03:23.002 INFO:teuthology.orchestra.run.smithi160.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 826 kB/s | 49 kB 00:00 2024-08-01T04:03:23.005 INFO:teuthology.orchestra.run.smithi003.stdout:(97/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-08-01T04:03:23.039 INFO:teuthology.orchestra.run.smithi003.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 164 kB/s | 47 kB 00:00 2024-08-01T04:03:23.050 INFO:teuthology.orchestra.run.smithi163.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-08-01T04:03:23.052 INFO:teuthology.orchestra.run.smithi160.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 5.8 MB/s | 151 kB 00:00 2024-08-01T04:03:23.072 INFO:teuthology.orchestra.run.smithi003.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-08-01T04:03:23.075 INFO:teuthology.orchestra.run.smithi163.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-08-01T04:03:23.086 INFO:teuthology.orchestra.run.smithi160.stdout:(112/160): libunwind-1.6.2-1.el9.x86_64.rpm 618 kB/s | 67 kB 00:00 2024-08-01T04:03:23.101 INFO:teuthology.orchestra.run.smithi163.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-08-01T04:03:23.114 INFO:teuthology.orchestra.run.smithi003.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 554 kB/s | 58 kB 00:00 2024-08-01T04:03:23.128 INFO:teuthology.orchestra.run.smithi160.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-08-01T04:03:23.162 INFO:teuthology.orchestra.run.smithi160.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-08-01T04:03:23.173 INFO:teuthology.orchestra.run.smithi003.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 963 kB/s | 96 kB 00:00 2024-08-01T04:03:23.187 INFO:teuthology.orchestra.run.smithi160.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-01T04:03:23.198 INFO:teuthology.orchestra.run.smithi003.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 513 kB/s | 64 kB 00:00 2024-08-01T04:03:23.213 INFO:teuthology.orchestra.run.smithi160.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-08-01T04:03:23.223 INFO:teuthology.orchestra.run.smithi003.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-08-01T04:03:23.229 INFO:teuthology.orchestra.run.smithi163.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 227 kB/s | 35 kB 00:00 2024-08-01T04:03:23.238 INFO:teuthology.orchestra.run.smithi160.stdout:(117/160): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-08-01T04:03:23.263 INFO:teuthology.orchestra.run.smithi160.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-08-01T04:03:23.297 INFO:teuthology.orchestra.run.smithi160.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-08-01T04:03:23.322 INFO:teuthology.orchestra.run.smithi160.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-08-01T04:03:23.347 INFO:teuthology.orchestra.run.smithi160.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-08-01T04:03:23.373 INFO:teuthology.orchestra.run.smithi160.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-08-01T04:03:23.393 INFO:teuthology.orchestra.run.smithi163.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-08-01T04:03:23.398 INFO:teuthology.orchestra.run.smithi160.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-08-01T04:03:23.418 INFO:teuthology.orchestra.run.smithi163.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 789 kB/s | 250 kB 00:00 2024-08-01T04:03:23.423 INFO:teuthology.orchestra.run.smithi160.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-08-01T04:03:23.424 INFO:teuthology.orchestra.run.smithi003.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 2.7 MB/s | 548 kB 00:00 2024-08-01T04:03:23.443 INFO:teuthology.orchestra.run.smithi163.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-08-01T04:03:23.448 INFO:teuthology.orchestra.run.smithi160.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-08-01T04:03:23.466 INFO:teuthology.orchestra.run.smithi003.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 7.4 MB/s | 309 kB 00:00 2024-08-01T04:03:23.473 INFO:teuthology.orchestra.run.smithi160.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-08-01T04:03:23.491 INFO:teuthology.orchestra.run.smithi003.stdout:(106/160): lua-devel-5.4.4-4.el9.x86_64.rpm 77 kB/s | 22 kB 00:00 2024-08-01T04:03:23.494 INFO:teuthology.orchestra.run.smithi163.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 777 kB/s | 77 kB 00:00 2024-08-01T04:03:23.499 INFO:teuthology.orchestra.run.smithi160.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-08-01T04:03:23.541 INFO:teuthology.orchestra.run.smithi160.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-08-01T04:03:23.544 INFO:teuthology.orchestra.run.smithi163.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-08-01T04:03:23.566 INFO:teuthology.orchestra.run.smithi160.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-01T04:03:23.569 INFO:teuthology.orchestra.run.smithi163.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-08-01T04:03:23.583 INFO:teuthology.orchestra.run.smithi160.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-08-01T04:03:23.591 INFO:teuthology.orchestra.run.smithi003.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-08-01T04:03:23.594 INFO:teuthology.orchestra.run.smithi163.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-08-01T04:03:23.610 INFO:teuthology.orchestra.run.smithi160.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-01T04:03:23.633 INFO:teuthology.orchestra.run.smithi003.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 734 kB/s | 26 kB 00:00 2024-08-01T04:03:23.642 INFO:teuthology.orchestra.run.smithi160.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-08-01T04:03:23.645 INFO:teuthology.orchestra.run.smithi163.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-08-01T04:03:23.667 INFO:teuthology.orchestra.run.smithi003.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-08-01T04:03:23.667 INFO:teuthology.orchestra.run.smithi160.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-08-01T04:03:23.670 INFO:teuthology.orchestra.run.smithi163.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-08-01T04:03:23.693 INFO:teuthology.orchestra.run.smithi160.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-08-01T04:03:23.695 INFO:teuthology.orchestra.run.smithi163.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-08-01T04:03:23.701 INFO:teuthology.orchestra.run.smithi003.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-08-01T04:03:23.718 INFO:teuthology.orchestra.run.smithi160.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-08-01T04:03:23.734 INFO:teuthology.orchestra.run.smithi003.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-08-01T04:03:23.735 INFO:teuthology.orchestra.run.smithi160.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-08-01T04:03:23.752 INFO:teuthology.orchestra.run.smithi160.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 918 kB/s | 31 kB 00:00 2024-08-01T04:03:23.777 INFO:teuthology.orchestra.run.smithi160.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-01T04:03:23.794 INFO:teuthology.orchestra.run.smithi160.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-08-01T04:03:23.819 INFO:teuthology.orchestra.run.smithi160.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 851 kB/s | 36 kB 00:00 2024-08-01T04:03:23.829 INFO:teuthology.orchestra.run.smithi163.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 263 kB/s | 42 kB 00:00 2024-08-01T04:03:23.838 INFO:teuthology.orchestra.run.smithi003.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 839 kB 00:00 2024-08-01T04:03:23.844 INFO:teuthology.orchestra.run.smithi160.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-01T04:03:23.854 INFO:teuthology.orchestra.run.smithi163.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 279 kB/s | 44 kB 00:00 2024-08-01T04:03:23.870 INFO:teuthology.orchestra.run.smithi160.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-01T04:03:23.902 INFO:teuthology.orchestra.run.smithi003.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 8.6 MB/s | 548 kB 00:00 2024-08-01T04:03:23.912 INFO:teuthology.orchestra.run.smithi160.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-08-01T04:03:23.927 INFO:teuthology.orchestra.run.smithi003.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-08-01T04:03:23.939 INFO:teuthology.orchestra.run.smithi160.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 731 kB/s | 20 kB 00:00 2024-08-01T04:03:23.952 INFO:teuthology.orchestra.run.smithi003.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-01T04:03:23.964 INFO:teuthology.orchestra.run.smithi160.stdout:(145/160): python3-werkzeug-2.0.3-3.el9.1.noarc 4.4 MB/s | 427 kB 00:00 2024-08-01T04:03:23.977 INFO:teuthology.orchestra.run.smithi003.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 583 kB/s | 14 kB 00:00 2024-08-01T04:03:23.990 INFO:teuthology.orchestra.run.smithi160.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-01T04:03:24.004 INFO:teuthology.orchestra.run.smithi163.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 243 kB/s | 36 kB 00:00 2024-08-01T04:03:24.019 INFO:teuthology.orchestra.run.smithi003.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-08-01T04:03:24.040 INFO:teuthology.orchestra.run.smithi160.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-08-01T04:03:24.070 INFO:teuthology.orchestra.run.smithi003.stdout:(118/160): protobuf-compiler-3.14.0-13.el9.x86_ 993 kB/s | 863 kB 00:00 2024-08-01T04:03:24.138 INFO:teuthology.orchestra.run.smithi163.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 715 kB/s | 221 kB 00:00 2024-08-01T04:03:24.163 INFO:teuthology.orchestra.run.smithi163.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 304 kB/s | 47 kB 00:00 2024-08-01T04:03:24.170 INFO:teuthology.orchestra.run.smithi003.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-08-01T04:03:24.212 INFO:teuthology.orchestra.run.smithi003.stdout:(120/160): python3-google-auth-2.32.0-1.el9.noa 5.9 MB/s | 216 kB 00:00 2024-08-01T04:03:24.232 INFO:teuthology.orchestra.run.smithi160.stdout:(148/160): librados2-19.1.0-1223.gf9911e17.el9. 14 MB/s | 3.4 MB 00:00 2024-08-01T04:03:24.254 INFO:teuthology.orchestra.run.smithi003.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-08-01T04:03:24.296 INFO:teuthology.orchestra.run.smithi003.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 3.5 MB/s | 144 kB 00:00 2024-08-01T04:03:24.307 INFO:teuthology.orchestra.run.smithi160.stdout:(149/160): librbd1-19.1.0-1223.gf9911e17.el9.x8 12 MB/s | 3.2 MB 00:00 2024-08-01T04:03:24.321 INFO:teuthology.orchestra.run.smithi003.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 443 kB/s | 11 kB 00:00 2024-08-01T04:03:24.338 INFO:teuthology.orchestra.run.smithi003.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 1.1 MB/s | 18 kB 00:00 2024-08-01T04:03:24.339 INFO:teuthology.orchestra.run.smithi163.stdout:(98/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-08-01T04:03:24.364 INFO:teuthology.orchestra.run.smithi003.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 957 kB/s | 23 kB 00:00 2024-08-01T04:03:24.364 INFO:teuthology.orchestra.run.smithi163.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-08-01T04:03:24.389 INFO:teuthology.orchestra.run.smithi003.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 805 kB/s | 19 kB 00:00 2024-08-01T04:03:24.389 INFO:teuthology.orchestra.run.smithi163.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 258 kB/s | 58 kB 00:00 2024-08-01T04:03:24.414 INFO:teuthology.orchestra.run.smithi003.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 814 kB/s | 20 kB 00:00 2024-08-01T04:03:24.439 INFO:teuthology.orchestra.run.smithi163.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 961 kB/s | 96 kB 00:00 2024-08-01T04:03:24.489 INFO:teuthology.orchestra.run.smithi163.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-08-01T04:03:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:(128/160): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-08-01T04:03:24.523 INFO:teuthology.orchestra.run.smithi163.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 334 kB/s | 45 kB 00:00 2024-08-01T04:03:24.540 INFO:teuthology.orchestra.run.smithi003.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.9 MB/s | 46 kB 00:00 2024-08-01T04:03:24.598 INFO:teuthology.orchestra.run.smithi163.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-08-01T04:03:24.632 INFO:teuthology.orchestra.run.smithi163.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-08-01T04:03:24.640 INFO:teuthology.orchestra.run.smithi003.stdout:(130/160): libarrow-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-08-01T04:03:24.657 INFO:teuthology.orchestra.run.smithi163.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-08-01T04:03:24.674 INFO:teuthology.orchestra.run.smithi003.stdout:(131/160): python3-more-itertools-8.12.0-2.el9. 594 kB/s | 79 kB 00:00 2024-08-01T04:03:24.699 INFO:teuthology.orchestra.run.smithi003.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-08-01T04:03:24.708 INFO:teuthology.orchestra.run.smithi163.stdout:(107/160): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-08-01T04:03:24.724 INFO:teuthology.orchestra.run.smithi003.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 683 kB/s | 16 kB 00:00 2024-08-01T04:03:24.758 INFO:teuthology.orchestra.run.smithi003.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.7 MB/s | 90 kB 00:00 2024-08-01T04:03:24.783 INFO:teuthology.orchestra.run.smithi163.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-08-01T04:03:24.792 INFO:teuthology.orchestra.run.smithi003.stdout:(135/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-08-01T04:03:24.817 INFO:teuthology.orchestra.run.smithi163.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-08-01T04:03:24.817 INFO:teuthology.orchestra.run.smithi003.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 486 kB/s | 28 kB 00:00 2024-08-01T04:03:24.842 INFO:teuthology.orchestra.run.smithi003.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 622 kB/s | 31 kB 00:00 2024-08-01T04:03:24.850 INFO:teuthology.orchestra.run.smithi163.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-08-01T04:03:24.884 INFO:teuthology.orchestra.run.smithi163.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-08-01T04:03:24.901 INFO:teuthology.orchestra.run.smithi003.stdout:(138/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-08-01T04:03:24.927 INFO:teuthology.orchestra.run.smithi003.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 714 kB/s | 59 kB 00:00 2024-08-01T04:03:24.952 INFO:teuthology.orchestra.run.smithi003.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 725 kB/s | 36 kB 00:00 2024-08-01T04:03:24.985 INFO:teuthology.orchestra.run.smithi003.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-08-01T04:03:25.010 INFO:teuthology.orchestra.run.smithi003.stdout:(142/160): python3-typing-extensions-4.12.2-2.e 1.3 MB/s | 74 kB 00:00 2024-08-01T04:03:25.026 INFO:teuthology.orchestra.run.smithi163.stdout:(112/160): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-08-01T04:03:25.068 INFO:teuthology.orchestra.run.smithi163.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-08-01T04:03:25.069 INFO:teuthology.orchestra.run.smithi003.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-08-01T04:03:25.101 INFO:teuthology.orchestra.run.smithi163.stdout:(114/160): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-08-01T04:03:25.118 INFO:teuthology.orchestra.run.smithi163.stdout:(115/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-08-01T04:03:25.119 INFO:teuthology.orchestra.run.smithi003.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-08-01T04:03:25.135 INFO:teuthology.orchestra.run.smithi163.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 954 kB/s | 32 kB 00:00 2024-08-01T04:03:25.144 INFO:teuthology.orchestra.run.smithi003.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-08-01T04:03:25.152 INFO:teuthology.orchestra.run.smithi163.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 421 kB/s | 14 kB 00:00 2024-08-01T04:03:25.174 INFO:teuthology.orchestra.run.smithi160.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-08-01T04:03:25.178 INFO:teuthology.orchestra.run.smithi163.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-08-01T04:03:25.186 INFO:teuthology.orchestra.run.smithi003.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 4.6 MB/s | 192 kB 00:00 2024-08-01T04:03:25.211 INFO:teuthology.orchestra.run.smithi163.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-08-01T04:03:25.228 INFO:teuthology.orchestra.run.smithi003.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-08-01T04:03:25.236 INFO:teuthology.orchestra.run.smithi163.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-08-01T04:03:25.261 INFO:teuthology.orchestra.run.smithi163.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-08-01T04:03:25.278 INFO:teuthology.orchestra.run.smithi163.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 8.5 MB/s | 144 kB 00:00 2024-08-01T04:03:25.300 INFO:teuthology.orchestra.run.smithi160.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 326 kB/s | 41 kB 00:00 2024-08-01T04:03:25.303 INFO:teuthology.orchestra.run.smithi163.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-08-01T04:03:25.329 INFO:teuthology.orchestra.run.smithi163.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-08-01T04:03:25.354 INFO:teuthology.orchestra.run.smithi163.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-08-01T04:03:25.379 INFO:teuthology.orchestra.run.smithi163.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-08-01T04:03:25.387 INFO:teuthology.orchestra.run.smithi003.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-08-01T04:03:25.404 INFO:teuthology.orchestra.run.smithi163.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-08-01T04:03:25.416 INFO:teuthology.orchestra.run.smithi160.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.1 MB/s | 247 kB 00:00 2024-08-01T04:03:25.455 INFO:teuthology.orchestra.run.smithi163.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-08-01T04:03:25.480 INFO:teuthology.orchestra.run.smithi163.stdout:(129/160): protobuf-compiler-3.14.0-13.el9.x86_ 872 kB/s | 863 kB 00:00 2024-08-01T04:03:25.505 INFO:teuthology.orchestra.run.smithi163.stdout:(130/160): python3-logutils-0.3.5-21.el9.noarch 923 kB/s | 46 kB 00:00 2024-08-01T04:03:25.525 INFO:teuthology.orchestra.run.smithi160.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 338 kB/s | 37 kB 00:00 2024-08-01T04:03:25.529 INFO:teuthology.orchestra.run.smithi003.stdout:(149/160): librados2-19.1.0-1223.gf9911e17.el9. 10 MB/s | 3.4 MB 00:00 2024-08-01T04:03:25.556 INFO:teuthology.orchestra.run.smithi163.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.5 MB/s | 58 kB 00:00 2024-08-01T04:03:25.598 INFO:teuthology.orchestra.run.smithi163.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-08-01T04:03:25.623 INFO:teuthology.orchestra.run.smithi163.stdout:(133/160): python3-more-itertools-8.12.0-2.el9. 553 kB/s | 79 kB 00:00 2024-08-01T04:03:25.634 INFO:teuthology.orchestra.run.smithi160.stdout:(154/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-08-01T04:03:25.650 INFO:teuthology.orchestra.run.smithi163.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.0 MB/s | 272 kB 00:00 2024-08-01T04:03:25.674 INFO:teuthology.orchestra.run.smithi163.stdout:(135/160): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-08-01T04:03:25.699 INFO:teuthology.orchestra.run.smithi163.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-08-01T04:03:25.724 INFO:teuthology.orchestra.run.smithi163.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-08-01T04:03:25.742 INFO:teuthology.orchestra.run.smithi160.stdout:(155/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-08-01T04:03:25.746 INFO:teuthology.orchestra.run.smithi003.stdout:(150/160): librbd1-19.1.0-1223.gf9911e17.el9.x8 6.1 MB/s | 3.2 MB 00:00 2024-08-01T04:03:25.749 INFO:teuthology.orchestra.run.smithi163.stdout:(138/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 713 kB/s | 90 kB 00:00 2024-08-01T04:03:25.775 INFO:teuthology.orchestra.run.smithi163.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-08-01T04:03:25.800 INFO:teuthology.orchestra.run.smithi163.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-08-01T04:03:25.825 INFO:teuthology.orchestra.run.smithi163.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-01T04:03:25.851 INFO:teuthology.orchestra.run.smithi163.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-08-01T04:03:25.867 INFO:teuthology.orchestra.run.smithi160.stdout:(156/160): openssl-3.2.2-2.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-08-01T04:03:25.876 INFO:teuthology.orchestra.run.smithi163.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-01T04:03:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:(151/160): NetworkManager-1.48.2-2.el9.x86_64.r 4.6 MB/s | 2.3 MB 00:00 2024-08-01T04:03:25.901 INFO:teuthology.orchestra.run.smithi163.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-01T04:03:25.926 INFO:teuthology.orchestra.run.smithi163.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-08-01T04:03:25.952 INFO:teuthology.orchestra.run.smithi163.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-01T04:03:25.963 INFO:teuthology.orchestra.run.smithi003.stdout:(152/160): NetworkManager-libnm-1.48.2-2.el9.x8 4.2 MB/s | 1.8 MB 00:00 2024-08-01T04:03:25.977 INFO:teuthology.orchestra.run.smithi163.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-01T04:03:25.996 INFO:teuthology.orchestra.run.smithi003.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.4 MB/s | 247 kB 00:00 2024-08-01T04:03:26.022 INFO:teuthology.orchestra.run.smithi003.stdout:(154/160): NetworkManager-team-1.48.2-2.el9.x86 148 kB/s | 41 kB 00:00 2024-08-01T04:03:26.027 INFO:teuthology.orchestra.run.smithi163.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-01T04:03:26.047 INFO:teuthology.orchestra.run.smithi003.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 442 kB/s | 37 kB 00:00 2024-08-01T04:03:26.081 INFO:teuthology.orchestra.run.smithi003.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.4 MB/s | 207 kB 00:00 2024-08-01T04:03:26.118 INFO:teuthology.orchestra.run.smithi160.stdout:(157/160): NetworkManager-libnm-1.48.2-2.el9.x8 1.0 MB/s | 1.8 MB 00:01 2024-08-01T04:03:26.173 INFO:teuthology.orchestra.run.smithi003.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-08-01T04:03:26.206 INFO:teuthology.orchestra.run.smithi003.stdout:(158/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.4 MB/s | 260 kB 00:00 2024-08-01T04:03:26.236 INFO:teuthology.orchestra.run.smithi163.stdout:(149/160): librados2-19.1.0-1223.gf9911e17.el9. 12 MB/s | 3.4 MB 00:00 2024-08-01T04:03:26.294 INFO:teuthology.orchestra.run.smithi163.stdout:(150/160): librbd1-19.1.0-1223.gf9911e17.el9.x8 10 MB/s | 3.2 MB 00:00 2024-08-01T04:03:26.298 INFO:teuthology.orchestra.run.smithi003.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 8.7 MB/s | 1.1 MB 00:00 2024-08-01T04:03:26.365 INFO:teuthology.orchestra.run.smithi003.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 8.5 MB/s | 2.4 MB 00:00 2024-08-01T04:03:26.370 INFO:teuthology.orchestra.run.smithi163.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 543 kB/s | 41 kB 00:00 2024-08-01T04:03:26.378 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-08-01T04:03:26.378 INFO:teuthology.orchestra.run.smithi003.stdout:Total 15 MB/s | 228 MB 00:14 2024-08-01T04:03:26.478 INFO:teuthology.orchestra.run.smithi163.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.2 MB/s | 247 kB 00:00 2024-08-01T04:03:26.545 INFO:teuthology.orchestra.run.smithi163.stdout:(153/160): NetworkManager-libnm-1.48.2-2.el9.x8 5.8 MB/s | 1.8 MB 00:00 2024-08-01T04:03:26.571 INFO:teuthology.orchestra.run.smithi163.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 399 kB/s | 37 kB 00:00 2024-08-01T04:03:26.612 INFO:teuthology.orchestra.run.smithi163.stdout:(155/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.9 MB/s | 2.3 MB 00:00 2024-08-01T04:03:26.637 INFO:teuthology.orchestra.run.smithi163.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-08-01T04:03:26.663 INFO:teuthology.orchestra.run.smithi163.stdout:(157/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.7 MB/s | 207 kB 00:00 2024-08-01T04:03:26.676 INFO:teuthology.orchestra.run.smithi160.stdout:(158/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 3.0 MB/s | 2.4 MB 00:00 2024-08-01T04:03:26.855 INFO:teuthology.orchestra.run.smithi163.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-08-01T04:03:26.910 INFO:teuthology.orchestra.run.smithi160.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 1.4 MB/s | 1.1 MB 00:00 2024-08-01T04:03:27.038 INFO:teuthology.orchestra.run.smithi163.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 6.0 MB/s | 2.4 MB 00:00 2024-08-01T04:03:27.072 INFO:teuthology.orchestra.run.smithi163.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 2.6 MB/s | 1.1 MB 00:00 2024-08-01T04:03:27.078 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-08-01T04:03:27.078 INFO:teuthology.orchestra.run.smithi163.stdout:Total 21 MB/s | 228 MB 00:10 2024-08-01T04:03:28.331 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-08-01T04:03:28.525 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-08-01T04:03:28.525 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-08-01T04:03:29.823 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-08-01T04:03:30.134 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-08-01T04:03:30.134 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-08-01T04:03:30.356 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-08-01T04:03:30.356 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-08-01T04:03:30.443 INFO:teuthology.orchestra.run.smithi160.stdout:(160/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-08-01T04:03:30.448 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-08-01T04:03:30.448 INFO:teuthology.orchestra.run.smithi160.stdout:Total 11 MB/s | 228 MB 00:20 2024-08-01T04:03:31.668 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-08-01T04:03:31.858 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-08-01T04:03:31.858 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-08-01T04:03:32.788 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-08-01T04:03:32.789 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-08-01T04:03:33.669 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-08-01T04:03:33.671 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-08-01T04:03:36.680 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-08-01T04:03:36.801 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-01T04:03:36.989 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-01T04:03:37.045 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-01T04:03:37.107 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-01T04:03:37.136 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-01T04:03:37.181 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-01T04:03:37.513 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-01T04:03:37.628 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:03:37.749 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:03:37.770 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:03:37.865 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:03:37.891 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 10/172 2024-08-01T04:03:37.924 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-01T04:03:38.122 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-01T04:03:38.181 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-01T04:03:38.209 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:03:38.313 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:03:38.344 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 15/172 2024-08-01T04:03:38.389 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 15/172 2024-08-01T04:03:38.411 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-01T04:03:38.431 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-01T04:03:38.620 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-01T04:03:38.710 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-01T04:03:38.882 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-01T04:03:39.043 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-01T04:03:39.105 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-01T04:03:39.214 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:39.297 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:39.499 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:39.606 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-01T04:03:39.697 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-08-01T04:03:39.701 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-01T04:03:39.736 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-01T04:03:39.771 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-01T04:03:39.789 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-01T04:03:39.845 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-01T04:03:39.867 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-01T04:03:39.897 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-01T04:03:39.909 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-08-01T04:03:39.936 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-01T04:03:39.981 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-01T04:03:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-01T04:03:39.988 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-01T04:03:40.013 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-01T04:03:40.026 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-01T04:03:40.041 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-01T04:03:40.044 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-01T04:03:40.082 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-01T04:03:40.096 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-01T04:03:40.111 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-01T04:03:40.130 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-01T04:03:40.156 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-01T04:03:40.164 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-01T04:03:40.190 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-01T04:03:40.194 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-01T04:03:40.198 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-01T04:03:40.238 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-01T04:03:40.315 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-01T04:03:40.348 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:19.1.0-1223.gf9911e17.el9. 36/172 2024-08-01T04:03:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el 37/172 2024-08-01T04:03:40.420 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_6 38/172 2024-08-01T04:03:40.491 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-01T04:03:40.521 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-01T04:03:40.565 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-01T04:03:40.624 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:03:40.625 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-01T04:03:40.643 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-01T04:03:40.721 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:03:40.729 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:03:40.758 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:03:40.780 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-01T04:03:40.838 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-01T04:03:40.853 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:03:40.879 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 10/172 2024-08-01T04:03:40.903 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:03:40.911 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-01T04:03:40.947 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:03:41.090 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:03:41.102 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-01T04:03:41.133 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 10/172 2024-08-01T04:03:41.160 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-01T04:03:41.188 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:03:41.206 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-01T04:03:41.292 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:03:41.314 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 15/172 2024-08-01T04:03:41.375 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-01T04:03:41.377 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 15/172 2024-08-01T04:03:41.399 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-01T04:03:41.418 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-01T04:03:41.451 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-01T04:03:41.478 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-01T04:03:41.514 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-01T04:03:41.525 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-01T04:03:41.555 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-01T04:03:41.556 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-01T04:03:41.588 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-01T04:03:41.593 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_6 49/172 2024-08-01T04:03:41.612 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:03:41.635 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-01T04:03:41.651 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-01T04:03:41.783 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-01T04:03:41.792 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:03:41.822 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 15/172 2024-08-01T04:03:41.829 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-01T04:03:41.837 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-01T04:03:41.893 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-01T04:03:41.917 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 15/172 2024-08-01T04:03:41.948 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-01T04:03:41.966 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-01T04:03:41.973 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-01T04:03:41.997 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-01T04:03:42.018 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-01T04:03:42.033 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-01T04:03:42.042 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-01T04:03:42.080 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-01T04:03:42.116 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-01T04:03:42.139 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:42.159 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-01T04:03:42.201 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-01T04:03:42.218 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:42.340 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-01T04:03:42.346 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-01T04:03:42.423 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:42.503 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-01T04:03:42.542 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-01T04:03:42.545 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-01T04:03:42.549 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-01T04:03:42.642 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-01T04:03:42.645 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-01T04:03:42.682 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-01T04:03:42.685 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-08-01T04:03:42.716 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-01T04:03:42.734 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-01T04:03:42.760 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-01T04:03:42.782 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-01T04:03:42.784 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-01T04:03:42.825 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-01T04:03:42.857 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-01T04:03:42.861 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-01T04:03:42.864 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-01T04:03:42.926 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-01T04:03:42.958 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-01T04:03:42.986 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-01T04:03:42.987 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-01T04:03:43.024 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-01T04:03:43.052 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-01T04:03:43.053 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-01T04:03:43.056 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-01T04:03:43.086 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-01T04:03:43.108 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-01T04:03:43.126 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-01T04:03:43.139 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-01T04:03:43.144 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-01T04:03:43.158 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-01T04:03:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-01T04:03:43.187 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-01T04:03:43.222 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-01T04:03:43.249 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-01T04:03:43.255 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-01T04:03:43.285 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:19.1.0-1223.gf9911e17.el9. 36/172 2024-08-01T04:03:43.297 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-01T04:03:43.320 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:43.331 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el 37/172 2024-08-01T04:03:43.339 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-01T04:03:43.374 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_6 38/172 2024-08-01T04:03:43.427 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-01T04:03:43.442 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-01T04:03:43.453 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:43.478 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-01T04:03:43.497 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-01T04:03:43.509 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-01T04:03:43.570 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-01T04:03:43.615 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-01T04:03:43.669 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-01T04:03:43.702 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-01T04:03:43.757 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-01T04:03:43.796 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-01T04:03:43.985 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-01T04:03:44.112 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-01T04:03:44.150 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-01T04:03:44.198 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-01T04:03:44.277 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-01T04:03:44.303 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-01T04:03:44.361 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-01T04:03:44.367 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-01T04:03:44.399 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-01T04:03:44.402 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-01T04:03:44.413 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-01T04:03:44.435 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-01T04:03:44.449 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-01T04:03:44.477 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-01T04:03:44.482 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-01T04:03:44.484 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-01T04:03:44.522 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_6 49/172 2024-08-01T04:03:44.550 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-01T04:03:44.564 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-01T04:03:44.585 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-01T04:03:44.591 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-01T04:03:44.638 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-01T04:03:44.682 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-01T04:03:44.708 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-01T04:03:44.716 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-01T04:03:44.767 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-01T04:03:44.772 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-01T04:03:44.802 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-01T04:03:44.802 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-01T04:03:44.821 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-01T04:03:44.844 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-01T04:03:44.893 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-01T04:03:44.911 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-01T04:03:44.926 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-01T04:03:44.962 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-01T04:03:44.970 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-01T04:03:44.999 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-01T04:03:45.008 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-01T04:03:45.027 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-01T04:03:45.044 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-01T04:03:45.062 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.1.0-1223.gf9911e17.el9. 36/172 2024-08-01T04:03:45.078 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-01T04:03:45.087 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-01T04:03:45.120 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el 37/172 2024-08-01T04:03:45.138 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-01T04:03:45.152 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_6 38/172 2024-08-01T04:03:45.272 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-01T04:03:45.276 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-01T04:03:45.323 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-01T04:03:45.331 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-01T04:03:45.364 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-01T04:03:45.427 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-01T04:03:45.470 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-01T04:03:45.506 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-08-01T04:03:45.545 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-01T04:03:45.571 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-01T04:03:45.596 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-01T04:03:45.611 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-01T04:03:45.662 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-01T04:03:45.684 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-01T04:03:45.735 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-01T04:03:45.790 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-01T04:03:45.828 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-01T04:03:45.849 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-01T04:03:45.856 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-01T04:03:45.890 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-01T04:03:45.930 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-01T04:03:45.961 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-01T04:03:45.991 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-01T04:03:46.025 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-01T04:03:46.058 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-01T04:03:46.100 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-01T04:03:46.126 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-01T04:03:46.213 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-01T04:03:46.256 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-01T04:03:46.288 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-01T04:03:46.390 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-01T04:03:46.409 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-01T04:03:46.438 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-01T04:03:46.508 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-01T04:03:46.542 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-01T04:03:46.580 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-01T04:03:46.644 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-01T04:03:46.691 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_6 49/172 2024-08-01T04:03:46.759 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-01T04:03:46.972 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-01T04:03:47.037 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-01T04:03:47.103 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-01T04:03:47.120 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-01T04:03:47.164 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-01T04:03:47.202 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-01T04:03:47.210 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-01T04:03:47.227 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-01T04:03:47.230 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-01T04:03:47.244 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-01T04:03:47.263 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-01T04:03:47.271 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-01T04:03:47.311 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-01T04:03:47.342 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-01T04:03:47.388 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-01T04:03:47.399 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-01T04:03:47.435 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-01T04:03:47.443 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-01T04:03:47.498 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-01T04:03:47.531 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-01T04:03:47.554 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-01T04:03:47.571 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-01T04:03:47.575 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-01T04:03:47.609 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 104/172 2024-08-01T04:03:47.696 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 104/172 2024-08-01T04:03:47.722 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 105/172 2024-08-01T04:03:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-01T04:03:47.750 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-01T04:03:47.772 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-01T04:03:47.777 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-01T04:03:47.801 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-01T04:03:47.853 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-01T04:03:47.863 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-01T04:03:47.984 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-01T04:03:48.026 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-08-01T04:03:48.078 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-01T04:03:48.182 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-01T04:03:48.182 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-01T04:03:48.247 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-01T04:03:48.275 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 108/172 2024-08-01T04:03:48.350 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-01T04:03:48.355 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 108/172 2024-08-01T04:03:48.384 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 109/172 2024-08-01T04:03:48.504 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-01T04:03:48.526 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-01T04:03:48.572 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-01T04:03:48.616 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-01T04:03:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-01T04:03:48.657 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-01T04:03:48.716 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-01T04:03:48.778 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-01T04:03:48.836 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-01T04:03:48.883 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-01T04:03:48.933 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-01T04:03:49.018 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-01T04:03:49.071 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-01T04:03:49.203 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-01T04:03:49.266 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-01T04:03:49.303 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-01T04:03:49.454 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-01T04:03:49.533 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-01T04:03:49.850 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-01T04:03:49.930 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-01T04:03:49.958 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-01T04:03:49.991 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-01T04:03:50.302 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-01T04:03:50.350 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 104/172 2024-08-01T04:03:50.447 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 104/172 2024-08-01T04:03:50.474 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 105/172 2024-08-01T04:03:50.497 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-01T04:03:50.519 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-01T04:03:50.557 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-01T04:03:50.615 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-01T04:03:50.677 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:50.706 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:50.747 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-01T04:03:50.808 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-01T04:03:50.872 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-01T04:03:50.996 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-01T04:03:51.005 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-01T04:03:51.031 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 108/172 2024-08-01T04:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-01T04:03:51.103 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 108/172 2024-08-01T04:03:51.133 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 109/172 2024-08-01T04:03:51.256 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-01T04:03:51.301 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 111/172 2024-08-01T04:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-01T04:03:51.395 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 111/172 2024-08-01T04:03:51.395 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-01T04:03:51.395 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-01T04:03:51.395 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:03:51.427 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 112/172 2024-08-01T04:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-01T04:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-01T04:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-01T04:03:53.410 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:53.445 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:54.032 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 111/172 2024-08-01T04:03:54.125 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 111/172 2024-08-01T04:03:54.125 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-01T04:03:54.125 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-01T04:03:54.125 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:03:54.158 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 112/172 2024-08-01T04:03:54.161 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-01T04:03:54.280 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-01T04:03:54.317 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-01T04:03:54.353 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-01T04:03:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-01T04:03:54.800 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 104/172 2024-08-01T04:03:54.929 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 104/172 2024-08-01T04:03:54.986 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 105/172 2024-08-01T04:03:55.052 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-01T04:03:55.639 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-01T04:03:55.670 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 108/172 2024-08-01T04:03:55.773 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 108/172 2024-08-01T04:03:55.806 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 109/172 2024-08-01T04:03:58.608 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:58.661 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:59.341 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 110/172 2024-08-01T04:03:59.389 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 111/172 2024-08-01T04:03:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 111/172 2024-08-01T04:03:59.554 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-01T04:03:59.554 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-01T04:03:59.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:03:59.598 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 112/172 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 112/172 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-08-01T04:04:03.240 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-08-01T04:04:03.241 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-08-01T04:04:03.241 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:03.412 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 113/172 2024-08-01T04:04:03.428 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 113/172 2024-08-01T04:04:03.428 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:03.428 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-01T04:04:03.428 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:04.942 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 114/172 2024-08-01T04:04:04.968 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 114/172 2024-08-01T04:04:04.968 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:04.968 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-01T04:04:04.968 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-01T04:04:04.968 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-01T04:04:04.968 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:05.201 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 115/172 2024-08-01T04:04:05.224 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 115/172 2024-08-01T04:04:05.224 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:05.224 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-01T04:04:05.224 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-01T04:04:05.224 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-01T04:04:05.224 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:05.660 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 116/172 2024-08-01T04:04:05.682 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 116/172 2024-08-01T04:04:05.682 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:05.682 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-01T04:04:05.683 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-01T04:04:05.683 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-01T04:04:05.683 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:05.790 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 112/172 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-08-01T04:04:05.840 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:05.847 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-01T04:04:05.902 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-01T04:04:06.012 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 113/172 2024-08-01T04:04:06.028 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 113/172 2024-08-01T04:04:06.028 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:06.028 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-01T04:04:06.028 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:06.090 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-01T04:04:06.136 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-01T04:04:06.214 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-01T04:04:06.304 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-01T04:04:06.453 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-01T04:04:06.503 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-01T04:04:06.535 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-01T04:04:06.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-01T04:04:07.033 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 128/172 2024-08-01T04:04:07.062 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 128/172 2024-08-01T04:04:07.093 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.e 129/172 2024-08-01T04:04:07.156 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17. 130/172 2024-08-01T04:04:07.185 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 131/172 2024-08-01T04:04:07.506 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 114/172 2024-08-01T04:04:07.533 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 114/172 2024-08-01T04:04:07.533 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:07.533 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-01T04:04:07.533 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-01T04:04:07.533 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-01T04:04:07.533 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:07.765 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 115/172 2024-08-01T04:04:07.788 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 115/172 2024-08-01T04:04:07.788 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:07.788 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-01T04:04:07.788 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-01T04:04:07.788 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-01T04:04:07.788 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:08.240 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 131/172 2024-08-01T04:04:08.243 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 116/172 2024-08-01T04:04:08.266 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 116/172 2024-08-01T04:04:08.266 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:08.266 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-01T04:04:08.266 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-01T04:04:08.266 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-01T04:04:08.266 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:08.279 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 132/172 2024-08-01T04:04:08.381 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-01T04:04:08.437 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-01T04:04:08.493 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-01T04:04:08.686 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-01T04:04:08.735 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-01T04:04:08.812 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-01T04:04:08.901 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-01T04:04:09.057 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-01T04:04:09.093 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-01T04:04:09.125 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-01T04:04:09.152 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-01T04:04:09.250 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 132/172 2024-08-01T04:04:09.271 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 133/172 2024-08-01T04:04:09.375 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 133/172 2024-08-01T04:04:09.480 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 128/172 2024-08-01T04:04:09.509 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 128/172 2024-08-01T04:04:09.541 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.e 129/172 2024-08-01T04:04:09.542 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el 134/172 2024-08-01T04:04:09.573 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 135/172 2024-08-01T04:04:09.597 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 135/172 2024-08-01T04:04:09.597 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:09.597 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-01T04:04:09.597 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-01T04:04:09.597 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-01T04:04:09.597 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:09.603 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17. 130/172 2024-08-01T04:04:09.632 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 131/172 2024-08-01T04:04:09.641 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 136/172 2024-08-01T04:04:09.653 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 136/172 2024-08-01T04:04:09.750 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 137/172 2024-08-01T04:04:09.806 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 138/172 2024-08-01T04:04:10.682 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 131/172 2024-08-01T04:04:10.718 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 132/172 2024-08-01T04:04:10.722 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-01T04:04:10.752 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 140/172 2024-08-01T04:04:10.775 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 140/172 2024-08-01T04:04:10.775 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:10.775 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-01T04:04:10.775 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-01T04:04:10.776 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-01T04:04:10.776 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:10.819 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1223.gf9911 141/172 2024-08-01T04:04:10.841 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1223.gf9911 141/172 2024-08-01T04:04:10.841 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:10.841 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-01T04:04:10.841 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:10.886 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 142/172 2024-08-01T04:04:10.909 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 142/172 2024-08-01T04:04:10.909 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:10.909 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-01T04:04:10.909 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-01T04:04:10.909 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-01T04:04:10.910 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:11.212 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 143/172 2024-08-01T04:04:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 143/172 2024-08-01T04:04:11.235 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:11.235 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-01T04:04:11.235 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-01T04:04:11.235 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-01T04:04:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:11.688 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 132/172 2024-08-01T04:04:11.710 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 133/172 2024-08-01T04:04:11.811 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 133/172 2024-08-01T04:04:11.988 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el 134/172 2024-08-01T04:04:12.020 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 135/172 2024-08-01T04:04:12.043 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 135/172 2024-08-01T04:04:12.043 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:12.043 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-01T04:04:12.043 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-01T04:04:12.043 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-01T04:04:12.043 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:12.087 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 136/172 2024-08-01T04:04:12.099 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 136/172 2024-08-01T04:04:12.197 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 137/172 2024-08-01T04:04:12.252 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 138/172 2024-08-01T04:04:13.043 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 112/172 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-08-01T04:04:13.044 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:13.166 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-01T04:04:13.198 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 140/172 2024-08-01T04:04:13.221 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 140/172 2024-08-01T04:04:13.221 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:13.222 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-01T04:04:13.222 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-01T04:04:13.222 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-01T04:04:13.222 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:13.265 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1223.gf9911 141/172 2024-08-01T04:04:13.270 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 113/172 2024-08-01T04:04:13.288 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1223.gf9911 141/172 2024-08-01T04:04:13.288 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:13.288 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-01T04:04:13.289 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:13.320 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 113/172 2024-08-01T04:04:13.320 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:13.320 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-01T04:04:13.320 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 142/172 2024-08-01T04:04:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 142/172 2024-08-01T04:04:13.355 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:13.355 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-01T04:04:13.356 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-01T04:04:13.356 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-01T04:04:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:13.658 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 143/172 2024-08-01T04:04:13.681 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 143/172 2024-08-01T04:04:13.681 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:13.681 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-01T04:04:13.681 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-01T04:04:13.681 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-01T04:04:13.681 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:15.022 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 114/172 2024-08-01T04:04:15.105 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 114/172 2024-08-01T04:04:15.106 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:15.106 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-01T04:04:15.106 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-01T04:04:15.106 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-01T04:04:15.106 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:15.381 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 115/172 2024-08-01T04:04:15.470 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 115/172 2024-08-01T04:04:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-01T04:04:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-01T04:04:15.470 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-01T04:04:15.471 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:15.783 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 144/172 2024-08-01T04:04:15.824 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 145/172 2024-08-01T04:04:15.865 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 146/172 2024-08-01T04:04:15.897 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-01T04:04:15.924 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-01T04:04:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 116/172 2024-08-01T04:04:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 116/172 2024-08-01T04:04:16.081 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:16.081 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-01T04:04:16.081 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-01T04:04:16.081 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-01T04:04:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:16.112 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 149/172 2024-08-01T04:04:16.176 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-01T04:04:16.201 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_ 151/172 2024-08-01T04:04:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-01T04:04:16.234 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-01T04:04:16.272 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-01T04:04:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-01T04:04:16.310 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-01T04:04:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-01T04:04:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-01T04:04:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-01T04:04:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-01T04:04:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-01T04:04:16.920 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:16.969 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:16.989 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-01T04:04:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-01T04:04:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-01T04:04:17.201 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-01T04:04:17.825 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 156/172 2024-08-01T04:04:17.869 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 128/172 2024-08-01T04:04:17.904 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 128/172 2024-08-01T04:04:17.907 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-01T04:04:17.948 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-01T04:04:17.958 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.e 129/172 2024-08-01T04:04:18.033 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-01T04:04:18.073 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17. 130/172 2024-08-01T04:04:18.089 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-01T04:04:18.105 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 131/172 2024-08-01T04:04:18.114 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-01T04:04:18.140 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-01T04:04:18.166 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-01T04:04:18.167 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-01T04:04:18.237 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 144/172 2024-08-01T04:04:18.250 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-01T04:04:18.275 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-01T04:04:18.275 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-01T04:04:18.287 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 145/172 2024-08-01T04:04:18.335 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 146/172 2024-08-01T04:04:18.360 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-01T04:04:18.367 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-01T04:04:18.385 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-01T04:04:18.390 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:18.390 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:18.394 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-01T04:04:18.576 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 149/172 2024-08-01T04:04:18.578 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:18.579 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-01T04:04:18.653 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-01T04:04:18.685 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_ 151/172 2024-08-01T04:04:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-01T04:04:18.721 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-01T04:04:18.726 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-01T04:04:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-01T04:04:18.755 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-01T04:04:18.802 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-01T04:04:18.811 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:19.430 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:19.468 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:19.470 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 131/172 2024-08-01T04:04:19.488 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:19.507 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 132/172 2024-08-01T04:04:20.332 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 156/172 2024-08-01T04:04:20.424 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-01T04:04:20.466 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-01T04:04:20.546 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-01T04:04:20.586 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-01T04:04:20.619 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-01T04:04:20.644 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-01T04:04:20.666 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 132/172 2024-08-01T04:04:20.671 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-01T04:04:20.671 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-01T04:04:20.690 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 133/172 2024-08-01T04:04:20.755 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-01T04:04:20.780 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-01T04:04:20.780 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-01T04:04:20.845 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 133/172 2024-08-01T04:04:20.864 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-01T04:04:20.889 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-01T04:04:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:20.894 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:21.039 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el 134/172 2024-08-01T04:04:21.076 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 135/172 2024-08-01T04:04:21.076 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:21.077 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-01T04:04:21.161 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-01T04:04:21.174 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 135/172 2024-08-01T04:04:21.174 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:21.174 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-01T04:04:21.174 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-01T04:04:21.174 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-01T04:04:21.175 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:21.193 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-01T04:04:21.228 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-01T04:04:21.235 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 136/172 2024-08-01T04:04:21.275 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 136/172 2024-08-01T04:04:21.292 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:21.453 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 137/172 2024-08-01T04:04:21.534 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 138/172 2024-08-01T04:04:22.626 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-01T04:04:22.662 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 140/172 2024-08-01T04:04:22.753 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 140/172 2024-08-01T04:04:22.753 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:22.753 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-01T04:04:22.754 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-01T04:04:22.754 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-01T04:04:22.754 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:22.804 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1223.gf9911 141/172 2024-08-01T04:04:22.902 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1223.gf9911 141/172 2024-08-01T04:04:22.902 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:22.902 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-01T04:04:22.902 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:22.982 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 142/172 2024-08-01T04:04:23.079 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 142/172 2024-08-01T04:04:23.079 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:23.079 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-01T04:04:23.079 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-01T04:04:23.079 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-01T04:04:23.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:23.464 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 143/172 2024-08-01T04:04:23.548 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 143/172 2024-08-01T04:04:23.548 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-01T04:04:23.548 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-01T04:04:23.548 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-01T04:04:23.548 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-01T04:04:23.548 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:24.488 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:24.488 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 1/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 2/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 3/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 4/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1223.gf9911 5/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 6/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 7/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 10/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 11/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 12/172 2024-08-01T04:04:24.489 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 13/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_ 15/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 16/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 17/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_6 18/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 19/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 20/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el 21/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:19.1.0-1223.gf9911e17.el9. 22/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_6 23/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 24/172 2024-08-01T04:04:24.490 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 25/172 2024-08-01T04:04:24.491 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 26/172 2024-08-01T04:04:24.491 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 27/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 28/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 29/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17. 30/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 31/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 32/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 33/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el 34/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 35/172 2024-08-01T04:04:24.503 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.e 36/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 37/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 38/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 39/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-01T04:04:24.504 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-01T04:04:24.505 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-01T04:04:24.506 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-01T04:04:24.507 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-01T04:04:24.508 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-01T04:04:24.509 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-08-01T04:04:24.510 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-01T04:04:24.511 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-01T04:04:24.512 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 149/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 151/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-01T04:04:24.513 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-01T04:04:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-01T04:04:26.562 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:26.562 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 1/172 2024-08-01T04:04:26.562 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 2/172 2024-08-01T04:04:26.562 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 3/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 4/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1223.gf9911 5/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 6/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 7/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 10/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 11/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 12/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 13/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_ 15/172 2024-08-01T04:04:26.563 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 16/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 17/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_6 18/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 19/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 20/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el 21/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:19.1.0-1223.gf9911e17.el9. 22/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_6 23/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 24/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 25/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 26/172 2024-08-01T04:04:26.564 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 27/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 28/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 29/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17. 30/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 31/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 32/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 33/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el 34/172 2024-08-01T04:04:26.577 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 35/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.e 36/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 37/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 38/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 39/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-01T04:04:26.578 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-01T04:04:26.579 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-01T04:04:26.580 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-01T04:04:26.581 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-01T04:04:26.582 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-01T04:04:26.583 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-01T04:04:26.584 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-01T04:04:26.585 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-01T04:04:26.586 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 149/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 151/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-01T04:04:26.587 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-01T04:04:26.588 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-01T04:04:27.293 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:27.293 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:27.293 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:27.294 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.295 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.296 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-01T04:04:27.297 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.298 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: lua-5.4.4-4.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-01T04:04:27.299 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.14-1.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-01T04:04:27.300 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-01T04:04:27.301 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-01T04:04:27.302 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-01T04:04:27.303 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-01T04:04:27.304 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-01T04:04:27.305 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-01T04:04:27.306 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-01T04:04:27.307 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-01T04:04:27.307 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:04:27.307 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-08-01T04:04:27.671 DEBUG:teuthology.parallel:result is None 2024-08-01T04:04:28.770 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 144/172 2024-08-01T04:04:28.824 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 145/172 2024-08-01T04:04:28.880 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 146/172 2024-08-01T04:04:28.919 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-01T04:04:28.957 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-01T04:04:29.261 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 149/172 2024-08-01T04:04:29.316 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-01T04:04:29.350 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_ 151/172 2024-08-01T04:04:29.397 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-01T04:04:29.447 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:29.490 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.2-8.el9.noarch 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.7.4-5.el9.x86_64 2024-08-01T04:04:29.491 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.4-9.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-01T04:04:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-01T04:04:29.495 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: lua-5.4.4-4.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-13.14-1.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-01T04:04:29.496 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-01T04:04:29.497 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-01T04:04:29.498 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-01T04:04:29.499 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-01T04:04:29.500 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-01T04:04:29.501 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-01T04:04:29.502 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: re2-1:20211101-3.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-01T04:04:29.503 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-01T04:04:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-01T04:04:29.504 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-01T04:04:29.504 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:04:29.504 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-08-01T04:04:29.893 DEBUG:teuthology.parallel:result is None 2024-08-01T04:04:30.555 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:30.601 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:30.661 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-01T04:04:32.071 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 156/172 2024-08-01T04:04:32.294 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-01T04:04:32.372 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-01T04:04:32.537 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-01T04:04:32.580 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-01T04:04:32.611 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-01T04:04:32.646 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-01T04:04:32.689 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-01T04:04:32.689 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-01T04:04:32.812 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-01T04:04:32.858 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-01T04:04:32.859 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-01T04:04:33.003 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-01T04:04:33.042 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-01T04:04:33.077 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:33.077 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:33.396 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-01T04:04:33.396 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-01T04:04:33.541 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-01T04:04:33.609 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-01T04:04:33.658 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-01T04:04:33.793 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:34.116 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:34.116 INFO:teuthology.orchestra.run.smithi003.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-01T04:04:34.117 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:39.659 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 1/172 2024-08-01T04:04:39.659 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 2/172 2024-08-01T04:04:39.659 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 3/172 2024-08-01T04:04:39.659 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 4/172 2024-08-01T04:04:39.659 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1223.gf9911 5/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 6/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 7/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 8/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 9/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 10/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 11/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 12/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 13/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 14/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_ 15/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 16/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 17/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_6 18/172 2024-08-01T04:04:39.660 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86 19/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 20/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el 21/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.1.0-1223.gf9911e17.el9. 22/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_6 23/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 24/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 25/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 26/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 27/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 28/172 2024-08-01T04:04:39.661 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 29/172 2024-08-01T04:04:39.675 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17. 30/172 2024-08-01T04:04:39.675 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noa 31/172 2024-08-01T04:04:39.675 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.n 32/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1223.gf99 33/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el 34/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 35/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.e 36/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 37/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 38/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 39/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-01T04:04:39.676 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-01T04:04:39.677 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-01T04:04:39.678 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-01T04:04:39.679 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-01T04:04:39.680 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-01T04:04:39.681 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-01T04:04:39.682 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-01T04:04:39.683 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-01T04:04:39.683 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-01T04:04:39.683 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-01T04:04:39.683 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-01T04:04:39.683 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-01T04:04:39.684 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-01T04:04:39.684 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-01T04:04:39.684 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-01T04:04:39.684 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-01T04:04:39.684 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-01T04:04:39.685 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-01T04:04:39.685 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-01T04:04:39.685 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-01T04:04:39.685 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-01T04:04:39.685 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-01T04:04:39.685 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-01T04:04:39.686 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-01T04:04:39.687 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 149/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 151/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-01T04:04:39.688 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-01T04:04:39.689 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-01T04:04:41.987 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-01T04:04:41.988 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-08-01T04:04:41.989 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.990 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.991 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.1.0-1223.gf9911e17.el9.noarch 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-01T04:04:41.992 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.993 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-01T04:04:41.994 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.14-1.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-01T04:04:41.995 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-01T04:04:41.996 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-01T04:04:41.997 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-01T04:04:41.998 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-01T04:04:41.999 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-01T04:04:42.000 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.1.0-1223.gf9911e17.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-3.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-01T04:04:42.001 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-01T04:04:42.002 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:04:42.002 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-08-01T04:04:42.630 DEBUG:teuthology.parallel:result is None 2024-08-01T04:04:42.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:04:42.771 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-01T04:04:42.827 INFO:teuthology.orchestra.run.smithi003.stdout:19.1.0-1223.gf9911e17.el9 2024-08-01T04:04:42.828 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1223.gf9911e17.el9 2024-08-01T04:04:42.828 INFO:teuthology.task.install:The correct ceph version 19.1.0-1223.gf9911e17 is installed. 2024-08-01T04:04:42.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:04:42.964 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-01T04:04:42.990 INFO:teuthology.orchestra.run.smithi160.stdout:19.1.0-1223.gf9911e17.el9 2024-08-01T04:04:42.990 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1223.gf9911e17.el9 2024-08-01T04:04:42.990 INFO:teuthology.task.install:The correct ceph version 19.1.0-1223.gf9911e17 is installed. 2024-08-01T04:04:42.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:04:43.127 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-01T04:04:43.152 INFO:teuthology.orchestra.run.smithi163.stdout:19.1.0-1223.gf9911e17.el9 2024-08-01T04:04:43.152 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1223.gf9911e17.el9 2024-08-01T04:04:43.153 INFO:teuthology.task.install:The correct ceph version 19.1.0-1223.gf9911e17 is installed. 2024-08-01T04:04:43.154 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-01T04:04:43.155 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:04:43.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-01T04:04:43.223 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:04:43.224 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-01T04:04:43.254 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:04:43.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-01T04:04:43.285 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-01T04:04:43.285 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:04:43.285 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-08-01T04:04:43.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-01T04:04:43.464 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:04:43.464 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-08-01T04:04:43.493 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-01T04:04:43.560 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:04:43.560 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-08-01T04:04:43.584 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-01T04:04:43.649 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-01T04:04:43.649 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:04:43.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-01T04:04:43.735 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-01T04:04:43.846 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:04:43.846 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-01T04:04:43.871 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-01T04:04:43.935 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:04:43.935 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-01T04:04:43.959 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-01T04:04:44.023 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-01T04:04:44.023 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:04:44.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-08-01T04:04:44.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-01T04:04:44.189 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:04:44.189 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-08-01T04:04:44.213 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-01T04:04:44.279 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:04:44.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-08-01T04:04:44.303 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-01T04:04:44.366 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-01T04:04:44.494 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776', 'subvols': {'create': 2}} 2024-08-01T04:04:44.495 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:04:44.495 INFO:tasks.cephadm:Cluster fsid is 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:04:44.495 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-01T04:04:44.496 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.160', 'mon.c': '172.21.15.163'} 2024-08-01T04:04:44.496 INFO:tasks.cephadm:First mon is mon.a on smithi003 2024-08-01T04:04:44.496 INFO:tasks.cephadm:First mgr is x 2024-08-01T04:04:44.496 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-01T04:04:44.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2024-08-01T04:04:44.562 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-08-01T04:04:44.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname $(hostname -s) 2024-08-01T04:04:44.609 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-01T04:04:44.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:04:44.746 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f9911e17b12222fbe598fff73cd3ad10d20c5776', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55731/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1223-gf9911e17', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1223.gf9911e17'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-30 22:03:49.797843', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-01T04:04:44.881 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 f9911e17b12222fbe598fff73cd3ad10d20c5776 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:04:44.882 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/x86_64/flavors/default/cephadm 2024-08-01T04:04:44.882 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/x86_64/flavors/default/cephadm 2024-08-01T04:04:44.883 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r--. 1 ubuntu ubuntu 784459 Aug 1 04:04 /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:45.481 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.008 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r--. 1 ubuntu ubuntu 784459 Aug 1 04:04 /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.008 DEBUG:teuthology.orchestra.run.smithi163:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/f9911e17b12222fbe598fff73cd3ad10d20c5776/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.549 INFO:teuthology.orchestra.run.smithi163.stdout:-rw-r--r--. 1 ubuntu ubuntu 784459 Aug 1 04:04 /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.550 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.606 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.625 DEBUG:teuthology.orchestra.run.smithi163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-01T04:04:46.659 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 on all hosts... 2024-08-01T04:04:46.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 pull 2024-08-01T04:04:46.663 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 pull 2024-08-01T04:04:46.667 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 pull 2024-08-01T04:04:47.502 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776... 2024-08-01T04:04:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776... 2024-08-01T04:04:48.158 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776... 2024-08-01T04:05:25.505 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-08-01T04:05:25.506 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_version": "ceph version 19.1.0-1223-gf9911e17 (f9911e17b12222fbe598fff73cd3ad10d20c5776) squid (rc)", 2024-08-01T04:05:25.506 INFO:teuthology.orchestra.run.smithi163.stdout: "image_id": "93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967", 2024-08-01T04:05:25.506 INFO:teuthology.orchestra.run.smithi163.stdout: "repo_digests": [ 2024-08-01T04:05:25.506 INFO:teuthology.orchestra.run.smithi163.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483" 2024-08-01T04:05:25.506 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2024-08-01T04:05:25.506 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.1.0-1223-gf9911e17 (f9911e17b12222fbe598fff73cd3ad10d20c5776) squid (rc)", 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967", 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483" 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-08-01T04:05:29.085 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.0-1223-gf9911e17 (f9911e17b12222fbe598fff73cd3ad10d20c5776) squid (rc)", 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967", 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483" 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:05:34.144 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:05:34.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2024-08-01T04:05:34.306 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-08-01T04:05:34.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph 2024-08-01T04:05:34.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2024-08-01T04:05:34.461 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-08-01T04:05:34.490 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /etc/ceph 2024-08-01T04:05:34.517 INFO:tasks.cephadm:Writing seed config... 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [client] debug client = 20 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-08-01T04:05:34.519 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-08-01T04:05:34.520 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-08-01T04:05:34.521 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-08-01T04:05:34.522 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-01T04:05:34.523 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-08-01T04:05:34.523 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-01T04:05:34.523 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-08-01T04:05:34.523 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:05:34.524 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-01T04:05:34.582 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-08-01T04:05:34.583 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service 2024-08-01T04:05:34.588 DEBUG:teuthology.orchestra.run.smithi003:mgr.x> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x.service 2024-08-01T04:05:34.634 INFO:tasks.cephadm:Bootstrapping... 2024-08-01T04:05:34.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 -v bootstrap --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:05:34.923 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-08-01T04:05:34.923 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776', '-v', 'bootstrap', '--fsid', '2f6d75a6-4fbb-11ef-bcc9-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2024-08-01T04:05:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-01T04:05:34.924 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2024-08-01T04:05:34.982 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.1.2 2024-08-01T04:05:34.982 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2024-08-01T04:05:34.982 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2024-08-01T04:05:34.998 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-01T04:05:34.998 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-01T04:05:35.017 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-01T04:05:35.017 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-08-01T04:05:35.040 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-08-01T04:05:35.059 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-08-01T04:05:35.059 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-08-01T04:05:35.059 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2024-08-01T04:05:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.1.2 2024-08-01T04:05:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 5.1.2 is present 2024-08-01T04:05:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2024-08-01T04:05:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2024-08-01T04:05:35.132 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-01T04:05:35.132 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-01T04:05:35.152 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-01T04:05:35.152 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-08-01T04:05:35.178 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-08-01T04:05:35.200 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-08-01T04:05:35.201 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-08-01T04:05:35.201 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2024-08-01T04:05:35.201 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:35.201 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 139752979900544 on /run/cephadm/2f6d75a6-4fbb-11ef-bcc9-c7b262605968.lock 2024-08-01T04:05:35.202 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 139752979900544 acquired on /run/cephadm/2f6d75a6-4fbb-11ef-bcc9-c7b262605968.lock 2024-08-01T04:05:35.203 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2024-08-01T04:05:35.204 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2024-08-01T04:05:35.205 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-08-01T04:05:35.220 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.3 metric 100 2024-08-01T04:05:35.220 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2024-08-01T04:05:35.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-01T04:05:35.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-08-01T04:05:35.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-08-01T04:05:35.245 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-01T04:05:35.245 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-01T04:05:35.246 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-01T04:05:35.246 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-08-01T04:05:35.246 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-08-01T04:05:35.246 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-01T04:05:35.248 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-08-01T04:05:35.249 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-08-01T04:05:35.249 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-01T04:05:35.252 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-01T04:05:35.254 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776... 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776... 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:44587d9a40a4d7d268e8b0379de4d23e82746b741d17871145a78a7892d49e6f 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:43735fbce9214f061a6894341d7822563f5aafe16fc7532189f4415cff6d2d91 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 2024-08-01T04:05:35.969 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-01T04:05:37.114 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 19.1.0-1223-gf9911e17 (f9911e17b12222fbe598fff73cd3ad10d20c5776) squid (rc) 2024-08-01T04:05:37.114 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 19.1.0-1223-gf9911e17 (f9911e17b12222fbe598fff73cd3ad10d20c5776) squid (rc) 2024-08-01T04:05:37.115 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2024-08-01T04:05:38.247 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2024-08-01T04:05:38.248 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2024-08-01T04:05:39.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQCSCatmQ8QmKxAAn664iFaxGBERSBgq45iwRA== 2024-08-01T04:05:40.437 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQCTCatmsqNvLxAAy4fDHMhfdDevz6xaYTnRPw== 2024-08-01T04:05:41.587 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQCUCatmTqZJOBAA78RQoA1+hGpjQq751lr+8A== 2024-08-01T04:05:41.589 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2024-08-01T04:05:42.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-01T04:05:42.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-01T04:05:42.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:42.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-01T04:05:42.683 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-01T04:05:42.684 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-08-01T04:05:42.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:42.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-01T04:05:42.684 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:05:42.684 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2024-08-01T04:05:43.883 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2024-08-01T04:05:44.264 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-01T04:05:44.509 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-01T04:05:44.796 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968.target → /etc/systemd/system/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968.target. 2024-08-01T04:05:44.797 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968.target → /etc/systemd/system/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968.target. 2024-08-01T04:05:45.156 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a 2024-08-01T04:05:45.157 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service: Unit ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service not loaded. 2024-08-01T04:05:45.445 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968.target.wants/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@.service. 2024-08-01T04:05:46.130 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:45 smithi003 podman[31882]: 2024-08-01 04:05:45.882916983 +0000 UTC m=+0.157171681 container create 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-01T04:05:46.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:46 smithi003 podman[31882]: 2024-08-01 04:05:46.128403131 +0000 UTC m=+0.402657836 container init 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-01T04:05:46.250 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-01T04:05:46.250 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-08-01T04:05:46.266 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-01T04:05:46.267 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-08-01T04:05:46.267 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-08-01T04:05:46.267 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-08-01T04:05:46.268 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2024-08-01T04:05:46.268 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2024-08-01T04:05:46.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:46 smithi003 podman[31882]: 2024-08-01 04:05:46.139312017 +0000 UTC m=+0.413566684 container start 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-01T04:05:46.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:46 smithi003 ceph-mon[31901]: mkfs 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:46.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:46 smithi003 bash[31882]: 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 2024-08-01T04:05:46.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:46 smithi003 ceph-mon[31901]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-01T04:05:46.414 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:46 smithi003 systemd[1]: Started Ceph mon.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:05:47.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-01T04:05:47.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: monmap epoch 1 2024-08-01T04:05:47.519 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:47.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: last_changed 2024-08-01T04:05:42.092880+0000 2024-08-01T04:05:47.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:05:47.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: min_mon_release 19 (squid) 2024-08-01T04:05:47.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: election_strategy: 1 2024-08-01T04:05:47.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:05:47.520 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: fsmap 2024-08-01T04:05:47.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: osdmap e1: 0 total, 0 up, 0 in 2024-08-01T04:05:47.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: mgrmap e1: no daemons active 2024-08-01T04:05:47.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:47 smithi003 ceph-mon[31901]: from='client.? 172.21.15.3:0/886780465' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-08-01T04:05:47.537 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2024-08-01T04:05:47.547 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.732416s) 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-01T04:05:47.548 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.862+0000 7f7859000640 1 Processor -- start 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.862+0000 7f7859000640 1 -- start start 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.862+0000 7f7859000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f7854109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:47.549 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.862+0000 7f7859000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f785410a280 con 0x7f78541098b0 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.863+0000 7f7853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f7854109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.864+0000 7f7853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f7854109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33458/0 (socket says 172.21.15.3:33458) 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.864+0000 7f7853400640 1 -- 172.21.15.3:0/378884069 learned_addr learned my addr 172.21.15.3:0/378884069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.864+0000 7f7853400640 1 -- 172.21.15.3:0/378884069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785410a720 con 0x7f78541098b0 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.866+0000 7f7853400640 1 --2- 172.21.15.3:0/378884069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f7854109cb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7844009820 tx=0x7f784402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705e8d46ddcd3da server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.867+0000 7f7852000640 1 -- 172.21.15.3:0/378884069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844037070 con 0x7f78541098b0 2024-08-01T04:05:47.550 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.867+0000 7f7852000640 1 -- 172.21.15.3:0/378884069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7844032440 con 0x7f78541098b0 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.867+0000 7f7852000640 1 -- 172.21.15.3:0/378884069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844032740 con 0x7f78541098b0 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.867+0000 7f7859000640 1 -- 172.21.15.3:0/378884069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 msgr2=0x7f7854109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.867+0000 7f7859000640 1 --2- 172.21.15.3:0/378884069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f7854109cb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7844009820 tx=0x7f784402a270 comp rx=0 tx=0).stop 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.868+0000 7f7859000640 1 -- 172.21.15.3:0/378884069 shutdown_connections 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.868+0000 7f7859000640 1 --2- 172.21.15.3:0/378884069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f7854109cb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.868+0000 7f7859000640 1 -- 172.21.15.3:0/378884069 >> 172.21.15.3:0/378884069 conn(0x7f785407cb50 msgr2=0x7f785407cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.868+0000 7f7859000640 1 -- 172.21.15.3:0/378884069 shutdown_connections 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.868+0000 7f7859000640 1 -- 172.21.15.3:0/378884069 wait complete. 2024-08-01T04:05:47.551 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.869+0000 7f7859000640 1 Processor -- start 2024-08-01T04:05:47.552 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.869+0000 7f7859000640 1 -- start start 2024-08-01T04:05:47.552 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.869+0000 7f7859000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f785419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:47.552 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.869+0000 7f7859000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f785410acc0 con 0x7f78541098b0 2024-08-01T04:05:47.552 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.870+0000 7f7853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f785419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:47.552 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.870+0000 7f7853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f785419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33466/0 (socket says 172.21.15.3:33466) 2024-08-01T04:05:47.552 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.870+0000 7f7853400640 1 -- 172.21.15.3:0/886780465 learned_addr learned my addr 172.21.15.3:0/886780465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.870+0000 7f7853400640 1 -- 172.21.15.3:0/886780465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785419d260 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.870+0000 7f7853400640 1 --2- 172.21.15.3:0/886780465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f785419cd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f78440090b0 tx=0x7f78440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844041070 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784403c7a0 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7844037040 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f7859000640 1 -- 172.21.15.3:0/886780465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785419d4f0 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f7859000640 1 -- 172.21.15.3:0/886780465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78541a01d0 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7844044070 con 0x7f78541098b0 2024-08-01T04:05:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.871+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f784403d730 con 0x7f78541098b0 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.872+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785410b340 con 0x7f78541098b0 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.876+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f784403cbe0 con 0x7f78541098b0 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.943+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7854062b20 con 0x7f78541098b0 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.944+0000 7f784be00640 1 -- 172.21.15.3:0/886780465 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f784403cdc0 con 0x7f78541098b0 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.945+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 msgr2=0x7f785419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.945+0000 7f7849600640 1 --2- 172.21.15.3:0/886780465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f785419cd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f78440090b0 tx=0x7f78440046c0 comp rx=0 tx=0).stop 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.946+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 shutdown_connections 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.946+0000 7f7849600640 1 --2- 172.21.15.3:0/886780465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78541098b0 0x7f785419cd20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.946+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 >> 172.21.15.3:0/886780465 conn(0x7f785407cb50 msgr2=0x7f7854190fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.946+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 shutdown_connections 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:46.946+0000 7f7849600640 1 -- 172.21.15.3:0/886780465 wait complete. 2024-08-01T04:05:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2024-08-01T04:05:47.555 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2024-08-01T04:05:48.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:48 smithi003 ceph-mon[31901]: from='client.? 172.21.15.3:0/642830640' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-01T04:05:48.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:48 smithi003 ceph-mon[31901]: from='client.? 172.21.15.3:0/642830640' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-01T04:05:48.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:05:48.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-08-01T04:05:48.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:48.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-08-01T04:05:48.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-01T04:05:48.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-01T04:05:48.835 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-01T04:05:48.835 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-01T04:05:48.835 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:05:48.835 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-01T04:05:48.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-01T04:05:48.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:05:48.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-08-01T04:05:48.836 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-08-01T04:05:48.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-01T04:05:48.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-01T04:05:48.837 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-01T04:05:48.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-01T04:05:48.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-01T04:05:48.838 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.122+0000 7fd18fe00640 1 Processor -- start 2024-08-01T04:05:48.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.122+0000 7fd18fe00640 1 -- start start 2024-08-01T04:05:48.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.123+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:48.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.123+0000 7fd18fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd19007c8e0 con 0x7fd19007deb0 2024-08-01T04:05:48.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.124+0000 7fd18ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:48.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.124+0000 7fd18ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33478/0 (socket says 172.21.15.3:33478) 2024-08-01T04:05:48.839 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.124+0000 7fd18ea00640 1 -- 172.21.15.3:0/1252784114 learned_addr learned my addr 172.21.15.3:0/1252784114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.124+0000 7fd18ea00640 1 -- 172.21.15.3:0/1252784114 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19007ca60 con 0x7fd19007deb0 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.125+0000 7fd18ea00640 1 --2- 172.21.15.3:0/1252784114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19007c310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd180009820 tx=0x7fd18002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c168f0f29bcb4a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.126+0000 7fd18e000640 1 -- 172.21.15.3:0/1252784114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd180037070 con 0x7fd19007deb0 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.126+0000 7fd18e000640 1 -- 172.21.15.3:0/1252784114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180032440 con 0x7fd19007deb0 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.126+0000 7fd18e000640 1 -- 172.21.15.3:0/1252784114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd180032740 con 0x7fd19007deb0 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.126+0000 7fd18fe00640 1 -- 172.21.15.3:0/1252784114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 msgr2=0x7fd19007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.126+0000 7fd18fe00640 1 --2- 172.21.15.3:0/1252784114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19007c310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd180009820 tx=0x7fd18002a270 comp rx=0 tx=0).stop 2024-08-01T04:05:48.841 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.127+0000 7fd18fe00640 1 -- 172.21.15.3:0/1252784114 shutdown_connections 2024-08-01T04:05:48.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.127+0000 7fd18fe00640 1 --2- 172.21.15.3:0/1252784114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19007c310 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:48.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.127+0000 7fd18fe00640 1 -- 172.21.15.3:0/1252784114 >> 172.21.15.3:0/1252784114 conn(0x7fd190102ac0 msgr2=0x7fd190104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:48.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.127+0000 7fd18fe00640 1 -- 172.21.15.3:0/1252784114 shutdown_connections 2024-08-01T04:05:48.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.127+0000 7fd18fe00640 1 -- 172.21.15.3:0/1252784114 wait complete. 2024-08-01T04:05:48.842 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.128+0000 7fd18fe00640 1 Processor -- start 2024-08-01T04:05:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.128+0000 7fd18fe00640 1 -- start start 2024-08-01T04:05:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.129+0000 7fd18fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.129+0000 7fd18fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd19007d000 con 0x7fd19007deb0 2024-08-01T04:05:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.129+0000 7fd18ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.129+0000 7fd18ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33480/0 (socket says 172.21.15.3:33480) 2024-08-01T04:05:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.129+0000 7fd18ea00640 1 -- 172.21.15.3:0/642830640 learned_addr learned my addr 172.21.15.3:0/642830640 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.129+0000 7fd18ea00640 1 -- 172.21.15.3:0/642830640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19019d270 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.130+0000 7fd18ea00640 1 --2- 172.21.15.3:0/642830640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19019cd30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd1800090b0 tx=0x7fd1800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.130+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd180041070 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.130+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18003c7a0 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.130+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd180037040 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.130+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19019d500 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.130+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901a01f0 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.131+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd180044070 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.131+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd18003d3f0 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.132+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19007c390 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.135+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fd180046450 con 0x7fd19007deb0 2024-08-01T04:05:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.201+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd19019d9e0 con 0x7fd19007deb0 2024-08-01T04:05:48.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.210+0000 7fd194a00640 1 -- 172.21.15.3:0/642830640 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+452 (secure 0 0 0) 0x7fd19019d9e0 con 0x7fd19007deb0 2024-08-01T04:05:48.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.211+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 msgr2=0x7fd19019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:48.846 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.211+0000 7fd18fe00640 1 --2- 172.21.15.3:0/642830640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19019cd30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd1800090b0 tx=0x7fd1800046c0 comp rx=0 tx=0).stop 2024-08-01T04:05:48.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.212+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 shutdown_connections 2024-08-01T04:05:48.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.212+0000 7fd18fe00640 1 --2- 172.21.15.3:0/642830640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd19007deb0 0x7fd19019cd30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:48.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.212+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 >> 172.21.15.3:0/642830640 conn(0x7fd190102ac0 msgr2=0x7fd190103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:48.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.212+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 shutdown_connections 2024-08-01T04:05:48.847 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:48.212+0000 7fd18fe00640 1 -- 172.21.15.3:0/642830640 wait complete. 2024-08-01T04:05:48.847 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2024-08-01T04:05:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:49 smithi003 ceph-mon[31901]: from='client.? 172.21.15.3:0/3194291494' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.419+0000 7f7b40e00640 1 Processor -- start 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.420+0000 7f7b40e00640 1 -- start start 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.420+0000 7f7b40e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c081640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.420+0000 7f7b40e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c081c10 con 0x7f7b3c081240 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.421+0000 7f7b3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c081640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.421+0000 7f7b3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c081640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33486/0 (socket says 172.21.15.3:33486) 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.421+0000 7f7b3b400640 1 -- 172.21.15.3:0/2192846840 learned_addr learned my addr 172.21.15.3:0/2192846840 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:50.165 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.421+0000 7f7b3b400640 1 -- 172.21.15.3:0/2192846840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c081d90 con 0x7f7b3c081240 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.422+0000 7f7b3b400640 1 --2- 172.21.15.3:0/2192846840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c081640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009820 tx=0x7f7b2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0948852acb4f560 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.423+0000 7f7b3aa00640 1 -- 172.21.15.3:0/2192846840 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c037070 con 0x7f7b3c081240 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.423+0000 7f7b3aa00640 1 -- 172.21.15.3:0/2192846840 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b2c032900 con 0x7f7b3c081240 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.423+0000 7f7b3aa00640 1 -- 172.21.15.3:0/2192846840 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c030850 con 0x7f7b3c081240 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.423+0000 7f7b40e00640 1 -- 172.21.15.3:0/2192846840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 msgr2=0x7f7b3c081640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.423+0000 7f7b40e00640 1 --2- 172.21.15.3:0/2192846840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c081640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009820 tx=0x7f7b2c02a270 comp rx=0 tx=0).stop 2024-08-01T04:05:50.166 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.424+0000 7f7b40e00640 1 -- 172.21.15.3:0/2192846840 shutdown_connections 2024-08-01T04:05:50.169 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.424+0000 7f7b40e00640 1 --2- 172.21.15.3:0/2192846840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c081640 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:50.169 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.424+0000 7f7b40e00640 1 -- 172.21.15.3:0/2192846840 >> 172.21.15.3:0/2192846840 conn(0x7f7b3c07cb50 msgr2=0x7f7b3c07ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:50.169 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.424+0000 7f7b40e00640 1 -- 172.21.15.3:0/2192846840 shutdown_connections 2024-08-01T04:05:50.169 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.424+0000 7f7b40e00640 1 -- 172.21.15.3:0/2192846840 wait complete. 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.425+0000 7f7b40e00640 1 Processor -- start 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.425+0000 7f7b40e00640 1 -- start start 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.425+0000 7f7b40e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.425+0000 7f7b40e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c111ef0 con 0x7f7b3c081240 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.426+0000 7f7b3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.426+0000 7f7b3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c1a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33496/0 (socket says 172.21.15.3:33496) 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.426+0000 7f7b3b400640 1 -- 172.21.15.3:0/3194291494 learned_addr learned my addr 172.21.15.3:0/3194291494 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.426+0000 7f7b3b400640 1 -- 172.21.15.3:0/3194291494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c1a1ec0 con 0x7f7b3c081240 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.426+0000 7f7b3b400640 1 --2- 172.21.15.3:0/3194291494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c1a1980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0097f0 tx=0x7f7b2c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.426+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c042070 con 0x7f7b3c081240 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.427+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b2c032ca0 con 0x7f7b3c081240 2024-08-01T04:05:50.170 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.427+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c037040 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.427+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3c1a2150 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.427+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3c1a4e40 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.427+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b2c045070 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.427+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b2c047a30 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.428+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3c112070 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.431+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f7b2c03da60 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.496+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7b3c1a53d0 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.497+0000 7f7b38c00640 1 -- 172.21.15.3:0/3194291494 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f7b2c03d430 con 0x7f7b3c081240 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.498+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 msgr2=0x7f7b3c1a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:50.171 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.498+0000 7f7b40e00640 1 --2- 172.21.15.3:0/3194291494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c1a1980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0097f0 tx=0x7f7b2c030ec0 comp rx=0 tx=0).stop 2024-08-01T04:05:50.172 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.499+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 shutdown_connections 2024-08-01T04:05:50.172 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.499+0000 7f7b40e00640 1 --2- 172.21.15.3:0/3194291494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b3c081240 0x7f7b3c1a1980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:50.172 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.499+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 >> 172.21.15.3:0/3194291494 conn(0x7f7b3c07cb50 msgr2=0x7f7b3c07d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:50.172 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.499+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 shutdown_connections 2024-08-01T04:05:50.172 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:49.499+0000 7f7b40e00640 1 -- 172.21.15.3:0/3194291494 wait complete. 2024-08-01T04:05:50.172 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2024-08-01T04:05:50.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:50 smithi003 systemd[1]: Stopping Ceph mon.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:05:50.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a[31897]: 2024-08-01T04:05:50.675+0000 7f30aac00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:05:50.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a[31897]: 2024-08-01T04:05:50.675+0000 7f30aac00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-01T04:05:50.989 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:50 smithi003 podman[32141]: 2024-08-01 04:05:50.769099795 +0000 UTC m=+0.213011913 container died 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:05:51.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 podman[32141]: 2024-08-01 04:05:51.098977228 +0000 UTC m=+0.542889355 container cleanup 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724) 2024-08-01T04:05:51.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 bash[32141]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a 2024-08-01T04:05:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 podman[32154]: 2024-08-01 04:05:51.29144806 +0000 UTC m=+0.505947027 container remove 4e73b4298aa2c6676ef4607588e06f8304a9d1db143816b975bdfbecbdd14bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724) 2024-08-01T04:05:52.218 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 systemd[1]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-01T04:05:52.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 systemd[1]: Stopped Ceph mon.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:05:52.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 systemd[1]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service: Consumed 2.073s CPU time. 2024-08-01T04:05:52.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:51 smithi003 systemd[1]: Starting Ceph mon.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:05:52.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 podman[32244]: 2024-08-01 04:05:52.123971252 +0000 UTC m=+0.044450468 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:05:52.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 podman[32244]: 2024-08-01 04:05:52.217447801 +0000 UTC m=+0.137927047 container create d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240724) 2024-08-01T04:05:52.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 podman[32244]: 2024-08-01 04:05:52.479226372 +0000 UTC m=+0.399705592 container init d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-01T04:05:52.580 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-01T04:05:52.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 podman[32244]: 2024-08-01 04:05:52.490966255 +0000 UTC m=+0.411445472 container start d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-01T04:05:52.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: set uid:gid to 167:167 (ceph:ceph) 2024-08-01T04:05:52.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: ceph version 19.1.0-1223-gf9911e17 (f9911e17b12222fbe598fff73cd3ad10d20c5776) squid (rc), process ceph-mon, pid 2 2024-08-01T04:05:52.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: pidfile_write: ignore empty --pid-file 2024-08-01T04:05:52.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: load: jerasure load: lrc 2024-08-01T04:05:52.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: RocksDB version: 7.9.2 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Git sha 0 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Compile date 2024-07-30 20:41:45 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: DB SUMMARY 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: DB Session ID: 91YRCLCZKMZ61MB0UD9W 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: CURRENT file: CURRENT 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: IDENTITY file: IDENTITY 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106262 ; 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.error_if_exists: 0 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.create_if_missing: 0 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.paranoid_checks: 1 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-01T04:05:52.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.env: 0x558028a7a7a0 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.fs: PosixFileSystem 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.info_log: 0x55805ffbd4a0 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_file_opening_threads: 16 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.statistics: (nil) 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.use_fsync: 0 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_log_file_size: 0 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.keep_log_file_num: 1000 2024-08-01T04:05:52.784 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.recycle_log_file_num: 0 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.allow_fallocate: 1 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.allow_mmap_reads: 0 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.allow_mmap_writes: 0 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.use_direct_reads: 0 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.create_missing_column_families: 0 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.db_log_dir: 2024-08-01T04:05:52.785 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.wal_dir: 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.advise_random_on_open: 1 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.db_write_buffer_size: 0 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.write_buffer_manager: 0x55805ffccdc0 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.rate_limiter: (nil) 2024-08-01T04:05:52.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.wal_recovery_mode: 2 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.enable_thread_tracking: 0 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.enable_pipelined_write: 0 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.unordered_write: 0 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.row_cache: None 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.wal_filter: None 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-01T04:05:52.787 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.allow_ingest_behind: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.two_write_queues: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.manual_wal_flush: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.wal_compression: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.atomic_flush: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-01T04:05:52.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.log_readahead_size: 0 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.best_efforts_recovery: 0 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.allow_data_in_errors: 0 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.db_host_id: __hostname__ 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_background_jobs: 2 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_background_compactions: -1 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_subcompactions: 1 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-01T04:05:52.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_total_wal_size: 0 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_open_files: -1 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bytes_per_sync: 0 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_readahead_size: 0 2024-08-01T04:05:52.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_background_flushes: -1 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Compression algorithms supported: 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kZSTD supported: 0 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kXpressCompression supported: 0 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kBZip2Compression supported: 0 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kLZ4Compression supported: 1 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kZlibCompression supported: 1 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kLZ4HCCompression supported: 1 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: kSnappyCompression supported: 1 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-01T04:05:52.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.merge_operator: 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_filter: None 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_filter_factory: None 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.sst_partitioner_factory: None 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55805ffbd0e0) 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2024-08-01T04:05:52.792 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x55805ffb11f0 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2024-08-01T04:05:52.793 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2024-08-01T04:05:52.794 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.write_buffer_size: 33554432 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_write_buffer_number: 2 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression: NoCompression 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression: Disabled 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.prefix_extractor: nullptr 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.num_levels: 7 2024-08-01T04:05:52.795 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-01T04:05:52.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-01T04:05:52.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-01T04:05:52.796 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.level: 32767 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.strategy: 0 2024-08-01T04:05:52.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.enabled: false 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.target_file_size_base: 67108864 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-01T04:05:52.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.arena_block_size: 1048576 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-01T04:05:52.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.disable_auto_compactions: 0 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.inplace_update_support: 0 2024-08-01T04:05:52.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.bloom_locality: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.max_successive_merges: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.paranoid_file_checks: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.force_consistency_checks: 1 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.report_bg_io_stats: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.ttl: 2592000 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-01T04:05:52.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.enable_blob_files: false 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.min_blob_size: 0 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.blob_file_size: 268435456 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.blob_file_starting_level: 0 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-01T04:05:52.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e65670d8-2cba-4887-aef8-0ea9341f4f64 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722485152545298, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722485152548684, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99115, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14548, "raw_average_key_size": 49, "raw_value_size": 91249, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722485152, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e65670d8-2cba-4887-aef8-0ea9341f4f64", "db_session_id": "91YRCLCZKMZ61MB0UD9W", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722485152548981, "job": 1, "event": "recovery_finished"} 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55805ffe0e00 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: DB pointer 0x5580600da000 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2024-08-01T04:05:52.803 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 100.72 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 100.72 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-01T04:05:52.804 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 44.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-01T04:05:52.805 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 9.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 9.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x55805ffb11f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???) e1 preinit fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).mds e1 new map 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).mds e1 print_map 2024-08-01T04:05:52.806 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout: btime 2024-08-01T04:05:46:214264+0000 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 bash[32244]: d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 2024-08-01T04:05:52.807 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 systemd[1]: Started Ceph mon.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: monmap epoch 1 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: last_changed 2024-08-01T04:05:42.092880+0000 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: min_mon_release 19 (squid) 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: election_strategy: 1 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: fsmap 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: osdmap e1: 0 total, 0 up, 0 in 2024-08-01T04:05:52.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:52 smithi003 ceph-mon[32262]: mgrmap e1: no daemons active 2024-08-01T04:05:54.001 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.250+0000 7f2cdea00640 1 Processor -- start 2024-08-01T04:05:54.001 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.251+0000 7f2cdea00640 1 -- start start 2024-08-01T04:05:54.001 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.251+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:54.002 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.251+0000 7f2cdea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd807c8c0 con 0x7f2cd807de90 2024-08-01T04:05:54.003 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.252+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:54.003 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.252+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57604/0 (socket says 172.21.15.3:57604) 2024-08-01T04:05:54.003 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.252+0000 7f2cd7e00640 1 -- 172.21.15.3:0/15487130 learned_addr learned my addr 172.21.15.3:0/15487130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:54.003 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.253+0000 7f2cd7e00640 1 -- 172.21.15.3:0/15487130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd807ca40 con 0x7f2cd807de90 2024-08-01T04:05:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.255+0000 7f2cd7e00640 1 --2- 172.21.15.3:0/15487130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd807c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009820 tx=0x7f2cc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6825f0b70526ecf9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.256+0000 7f2cd6a00640 1 -- 172.21.15.3:0/15487130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8037070 con 0x7f2cd807de90 2024-08-01T04:05:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.256+0000 7f2cd6a00640 1 -- 172.21.15.3:0/15487130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2cc8032900 con 0x7f2cd807de90 2024-08-01T04:05:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.256+0000 7f2cd6a00640 1 -- 172.21.15.3:0/15487130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8030850 con 0x7f2cd807de90 2024-08-01T04:05:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 -- 172.21.15.3:0/15487130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 msgr2=0x7f2cd807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 --2- 172.21.15.3:0/15487130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd807c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009820 tx=0x7f2cc802a270 comp rx=0 tx=0).stop 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 -- 172.21.15.3:0/15487130 shutdown_connections 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 --2- 172.21.15.3:0/15487130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd807c2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 -- 172.21.15.3:0/15487130 >> 172.21.15.3:0/15487130 conn(0x7f2cd8102d00 msgr2=0x7f2cd8105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 -- 172.21.15.3:0/15487130 shutdown_connections 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.257+0000 7f2cdea00640 1 -- 172.21.15.3:0/15487130 wait complete. 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cdea00640 1 Processor -- start 2024-08-01T04:05:54.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cdea00640 1 -- start start 2024-08-01T04:05:54.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cdea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd810dba0 con 0x7f2cd807de90 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd819d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57620/0 (socket says 172.21.15.3:57620) 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.259+0000 7f2cd7e00640 1 -- 172.21.15.3:0/65835794 learned_addr learned my addr 172.21.15.3:0/65835794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.260+0000 7f2cd7e00640 1 -- 172.21.15.3:0/65835794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd819dc10 con 0x7f2cd807de90 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.260+0000 7f2cd7e00640 1 --2- 172.21.15.3:0/65835794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd819d6d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80097f0 tx=0x7f2cc8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.260+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8037040 con 0x7f2cd807de90 2024-08-01T04:05:54.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.261+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2cc8040070 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.261+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd819dea0 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.261+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cc803a390 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.261+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd81a0b90 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.261+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2cc803a5e0 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.261+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2cc8045950 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.262+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd807c370 con 0x7f2cd807de90 2024-08-01T04:05:54.010 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.265+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f2cc8045460 con 0x7f2cd807de90 2024-08-01T04:05:54.011 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.330+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f2cd819e320 con 0x7f2cd807de90 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.334+0000 7f2cd4c00640 1 -- 172.21.15.3:0/65835794 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f2cc803b430 con 0x7f2cd807de90 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.335+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 msgr2=0x7f2cd819d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.335+0000 7f2cdea00640 1 --2- 172.21.15.3:0/65835794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd819d6d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80097f0 tx=0x7f2cc8030d80 comp rx=0 tx=0).stop 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.335+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 shutdown_connections 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.335+0000 7f2cdea00640 1 --2- 172.21.15.3:0/65835794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cd807de90 0x7f2cd819d6d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.335+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 >> 172.21.15.3:0/65835794 conn(0x7f2cd8102d00 msgr2=0x7f2cd8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.336+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 shutdown_connections 2024-08-01T04:05:54.012 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:53.336+0000 7f2cdea00640 1 -- 172.21.15.3:0/65835794 wait complete. 2024-08-01T04:05:54.013 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-01T04:05:54.035 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:05:54.035 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2024-08-01T04:05:54.037 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-01T04:05:54.038 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-01T04:05:54.497 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x 2024-08-01T04:05:54.497 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x.service: Unit ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x.service not loaded. 2024-08-01T04:05:54.761 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968.target.wants/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@.service. 2024-08-01T04:05:54.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:54 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/65835794' entity='client.admin' 2024-08-01T04:05:55.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:54 smithi003 systemd[1]: Starting Ceph mgr.x for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:05:55.482 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 podman[32472]: 2024-08-01 04:05:55.082716205 +0000 UTC m=+0.042613127 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 2024-08-01T04:05:55.482 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 podman[32472]: 2024-08-01 04:05:55.194683342 +0000 UTC m=+0.154580271 container create ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-01T04:05:55.482 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 podman[32472]: 2024-08-01 04:05:55.406375675 +0000 UTC m=+0.366272611 container init ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:05:55.483 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 podman[32472]: 2024-08-01 04:05:55.417583121 +0000 UTC m=+0.377480056 container start ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-08-01T04:05:55.483 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 bash[32472]: ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c 2024-08-01T04:05:55.512 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-01T04:05:55.512 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-08-01T04:05:55.529 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-01T04:05:55.529 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-08-01T04:05:55.529 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-08-01T04:05:55.529 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-08-01T04:05:55.549 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-01T04:05:55.549 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-08-01T04:05:55.565 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-01T04:05:55.565 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-08-01T04:05:55.565 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-08-01T04:05:55.566 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-01T04:05:55.566 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2024-08-01T04:05:55.566 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2024-08-01T04:05:55.740 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 systemd[1]: Started Ceph mgr.x for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:05:55.740 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:55.622+0000 7f4c787d4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:05:56.027 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:55.821+0000 7f4c787d4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:05:56.027 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:55 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:55.911+0000 7f4c787d4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:05:56.284 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:56 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:56.228+0000 7f4c787d4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:05:56.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:05:56 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3633086822' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-01T04:05:56.548 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:56 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:56.305+0000 7f4c787d4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:05:58.205 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:57 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:57.730+0000 7f4c787d4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:05:58.205 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:57 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:57.871+0000 7f4c787d4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:05:58.205 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:57 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:57.944+0000 7f4c787d4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:05:58.205 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:58.202+0000 7f4c787d4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:05:58.535 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:58.278+0000 7f4c787d4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:05:58.535 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:58.359+0000 7f4c787d4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:05:58.536 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:58.532+0000 7f4c787d4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:05:59.463 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:59 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:59.083+0000 7f4c787d4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:05:59.723 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:59 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:59.460+0000 7f4c787d4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:05:59.723 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:59 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:59.541+0000 7f4c787d4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:05:59.723 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:05:59 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:05:59.719+0000 7f4c787d4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:06:00.458 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.095+0000 7f4c787d4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:06:00.459 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.170+0000 7f4c787d4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:06:00.459 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.408+0000 7f4c787d4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:06:00.787 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.605+0000 7f4c787d4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:06:00.788 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.684+0000 7f4c787d4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:06:00.788 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.784+0000 7f4c787d4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:06:01.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:00.858+0000 7f4c787d4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:06:01.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:01.507+0000 7f4c787d4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:06:01.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:01.694+0000 7f4c787d4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: Activating manager daemon x 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: mgrmap e2: x(active, starting, since 0.0123224s) 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: Manager daemon x is now available 2024-08-01T04:06:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:06:01.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' 2024-08-01T04:06:01.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:06:01.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' 2024-08-01T04:06:01.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:01 smithi003 ceph-mon[32262]: from='mgr.14100 172.21.15.3:0/2819354271' entity='mgr.x' 2024-08-01T04:06:03.977 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:03 smithi003 ceph-mon[32262]: mgrmap e3: x(active, since 1.02113s) 2024-08-01T04:06:05.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:04 smithi003 ceph-mon[32262]: mgrmap e4: x(active, since 2s) 2024-08-01T04:06:05.275 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:06:05.277 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-08-01T04:06:05.277 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:06:05.277 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-08-01T04:06:05.277 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-01T04:06:05.277 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-01T04:06:05.279 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-01T04:06:05.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-01T04:06:05.287 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.288 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-08-01T04:05:46:214264+0000", 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-01T04:06:05.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-08-01T04:05:46.215159+0000", 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-08-01T04:06:05.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.152+0000 7fed4c400640 1 Processor -- start 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.152+0000 7fed4c400640 1 -- start start 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.152+0000 7fed4c400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed44109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.152+0000 7fed4c400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410a280 con 0x7fed441098b0 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.153+0000 7fed49a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed44109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.153+0000 7fed49a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed44109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57660/0 (socket says 172.21.15.3:57660) 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.153+0000 7fed49a00640 1 -- 172.21.15.3:0/191645179 learned_addr learned my addr 172.21.15.3:0/191645179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.153+0000 7fed49a00640 1 -- 172.21.15.3:0/191645179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4410a920 con 0x7fed441098b0 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.155+0000 7fed49a00640 1 --2- 172.21.15.3:0/191645179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed44109cb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fed38009820 tx=0x7fed3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e34c8941de70d99 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:05.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.155+0000 7fed43e00640 1 -- 172.21.15.3:0/191645179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed38037070 con 0x7fed441098b0 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.155+0000 7fed43e00640 1 -- 172.21.15.3:0/191645179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed38032900 con 0x7fed441098b0 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.155+0000 7fed43e00640 1 -- 172.21.15.3:0/191645179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed38030850 con 0x7fed441098b0 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 -- 172.21.15.3:0/191645179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 msgr2=0x7fed44109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 --2- 172.21.15.3:0/191645179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed44109cb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fed38009820 tx=0x7fed3802a270 comp rx=0 tx=0).stop 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 -- 172.21.15.3:0/191645179 shutdown_connections 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 --2- 172.21.15.3:0/191645179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed44109cb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 -- 172.21.15.3:0/191645179 >> 172.21.15.3:0/191645179 conn(0x7fed4407cdc0 msgr2=0x7fed4407d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 -- 172.21.15.3:0/191645179 shutdown_connections 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.156+0000 7fed4c400640 1 -- 172.21.15.3:0/191645179 wait complete. 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.157+0000 7fed4c400640 1 Processor -- start 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.158+0000 7fed4c400640 1 -- start start 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.158+0000 7fed4c400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed4419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:05.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.158+0000 7fed4c400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed4410dcd0 con 0x7fed441098b0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.158+0000 7fed49a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed4419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.158+0000 7fed49a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed4419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57664/0 (socket says 172.21.15.3:57664) 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.158+0000 7fed49a00640 1 -- 172.21.15.3:0/3633086822 learned_addr learned my addr 172.21.15.3:0/3633086822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed49a00640 1 -- 172.21.15.3:0/3633086822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419dc80 con 0x7fed441098b0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed49a00640 1 --2- 172.21.15.3:0/3633086822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed4419d740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed380097f0 tx=0x7fed38030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed38042070 con 0x7fed441098b0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed38032ca0 con 0x7fed441098b0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed38037040 con 0x7fed441098b0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4419df10 con 0x7fed441098b0 2024-08-01T04:06:05.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.159+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a0c00 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.160+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed38045070 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.160+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed38047a30 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.160+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4410de50 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.163+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fed3803da60 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.230+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fed44109d30 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.231+0000 7fed42000640 1 -- 172.21.15.3:0/3633086822 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fed44109d30 con 0x7fed441098b0 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 msgr2=0x7fed4419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 --2- 172.21.15.3:0/3633086822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed4419d740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed380097f0 tx=0x7fed38030ec0 comp rx=0 tx=0).stop 2024-08-01T04:06:05.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 shutdown_connections 2024-08-01T04:06:05.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 --2- 172.21.15.3:0/3633086822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed441098b0 0x7fed4419d740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:05.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 >> 172.21.15.3:0/3633086822 conn(0x7fed4407cdc0 msgr2=0x7fed44107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:05.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 shutdown_connections 2024-08-01T04:06:05.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:05:56.232+0000 7fed4c400640 1 -- 172.21.15.3:0/3633086822 wait complete. 2024-08-01T04:06:05.295 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2024-08-01T04:06:08.518 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:08 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3376539899' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-01T04:06:08.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-01T04:06:08.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:08.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-01T04:06:08.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-01T04:06:08.819 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-01T04:06:08.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-01T04:06:08.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-01T04:06:08.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-01T04:06:08.827 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.828 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-08-01T04:05:46:214264+0000", 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-01T04:06:08.829 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-08-01T04:05:46.215159+0000", 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-08-01T04:06:08.830 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.871+0000 7fb48e600640 1 Processor -- start 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.872+0000 7fb48e600640 1 -- start start 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.872+0000 7fb48e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb488109c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.872+0000 7fb48e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810a230 con 0x7fb488109860 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.873+0000 7fb48d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb488109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.873+0000 7fb48d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb488109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50284/0 (socket says 172.21.15.3:50284) 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.873+0000 7fb48d200640 1 -- 172.21.15.3:0/3805394703 learned_addr learned my addr 172.21.15.3:0/3805394703 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.873+0000 7fb48d200640 1 -- 172.21.15.3:0/3805394703 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48810a740 con 0x7fb488109860 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.874+0000 7fb48d200640 1 --2- 172.21.15.3:0/3805394703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb488109c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009820 tx=0x7fb47c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e2b826f9c5854c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.875+0000 7fb487400640 1 -- 172.21.15.3:0/3805394703 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb47c037070 con 0x7fb488109860 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.875+0000 7fb487400640 1 -- 172.21.15.3:0/3805394703 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb47c032900 con 0x7fb488109860 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.875+0000 7fb487400640 1 -- 172.21.15.3:0/3805394703 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb47c030850 con 0x7fb488109860 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.875+0000 7fb48e600640 1 -- 172.21.15.3:0/3805394703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 msgr2=0x7fb488109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:08.831 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.875+0000 7fb48e600640 1 --2- 172.21.15.3:0/3805394703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb488109c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009820 tx=0x7fb47c02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.876+0000 7fb48e600640 1 -- 172.21.15.3:0/3805394703 shutdown_connections 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.876+0000 7fb48e600640 1 --2- 172.21.15.3:0/3805394703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb488109c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.876+0000 7fb48e600640 1 -- 172.21.15.3:0/3805394703 >> 172.21.15.3:0/3805394703 conn(0x7fb48807cb20 msgr2=0x7fb48807cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.876+0000 7fb48e600640 1 -- 172.21.15.3:0/3805394703 shutdown_connections 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.876+0000 7fb48e600640 1 -- 172.21.15.3:0/3805394703 wait complete. 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.877+0000 7fb48e600640 1 Processor -- start 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.877+0000 7fb48e600640 1 -- start start 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.877+0000 7fb48e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb48819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.877+0000 7fb48e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810daf0 con 0x7fb488109860 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.878+0000 7fb48d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb48819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.878+0000 7fb48d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb48819d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50288/0 (socket says 172.21.15.3:50288) 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.878+0000 7fb48d200640 1 -- 172.21.15.3:0/3376539899 learned_addr learned my addr 172.21.15.3:0/3376539899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:08.832 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.878+0000 7fb48d200640 1 -- 172.21.15.3:0/3376539899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48819da80 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.878+0000 7fb48d200640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb48819d540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb47c030460 tx=0x7fb47c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.879+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb47c042070 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.879+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb47c032be0 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.879+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb47c037040 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.879+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48819dd10 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.879+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881a0a00 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.880+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb47c045070 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.880+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb488109c60 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.881+0000 7fb485600640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb470038f70 0x7fb47003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.881+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb47c06bd70 con 0x7fb488109860 2024-08-01T04:06:08.833 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.881+0000 7fb487e00640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb470038f70 0x7fb47003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.882+0000 7fb487e00640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb470038f70 0x7fb47003b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4780098c0 tx=0x7fb478006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:07.887+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb47c045360 con 0x7fb488109860 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.154+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4881a1080 con 0x7fb488109860 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.155+0000 7fb485600640 1 -- 172.21.15.3:0/3376539899 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb47c06b790 con 0x7fb488109860 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb470038f70 msgr2=0x7fb47003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb470038f70 0x7fb47003b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4780098c0 tx=0x7fb478006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 msgr2=0x7fb48819d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb48819d540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb47c030460 tx=0x7fb47c030d80 comp rx=0 tx=0).stop 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 shutdown_connections 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb470038f70 0x7fb47003b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 --2- 172.21.15.3:0/3376539899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb488109860 0x7fb48819d540 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:08.834 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 >> 172.21.15.3:0/3376539899 conn(0x7fb48807cb20 msgr2=0x7fb4881074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:08.835 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.158+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 shutdown_connections 2024-08-01T04:06:08.835 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:08.159+0000 7fb48e600640 1 -- 172.21.15.3:0/3376539899 wait complete. 2024-08-01T04:06:08.835 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2024-08-01T04:06:10.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:09 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/783205055' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-01T04:06:10.418 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:06:10.418 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-08-01T04:06:10.418 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:06:10.418 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-08-01T04:06:10.418 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-01T04:06:10.418 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-01T04:06:10.419 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.376+0000 7f7538a00640 1 Processor -- start 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.376+0000 7f7538a00640 1 -- start start 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.376+0000 7f7538a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f7534107770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.376+0000 7f7538a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7534107d40 con 0x7f7534107370 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.377+0000 7f7533400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f7534107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.377+0000 7f7533400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f7534107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50296/0 (socket says 172.21.15.3:50296) 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.377+0000 7f7533400640 1 -- 172.21.15.3:0/1787738603 learned_addr learned my addr 172.21.15.3:0/1787738603 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.377+0000 7f7533400640 1 -- 172.21.15.3:0/1787738603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7534108520 con 0x7f7534107370 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.378+0000 7f7533400640 1 --2- 172.21.15.3:0/1787738603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f7534107770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7524009820 tx=0x7f752402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4f14e1b08e7eb4 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.378+0000 7f7532000640 1 -- 172.21.15.3:0/1787738603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7524037070 con 0x7f7534107370 2024-08-01T04:06:10.420 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7532000640 1 -- 172.21.15.3:0/1787738603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7524032900 con 0x7f7534107370 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7532000640 1 -- 172.21.15.3:0/1787738603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7524030850 con 0x7f7534107370 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 -- 172.21.15.3:0/1787738603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 msgr2=0x7f7534107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 --2- 172.21.15.3:0/1787738603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f7534107770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7524009820 tx=0x7f752402a270 comp rx=0 tx=0).stop 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 -- 172.21.15.3:0/1787738603 shutdown_connections 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 --2- 172.21.15.3:0/1787738603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f7534107770 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 -- 172.21.15.3:0/1787738603 >> 172.21.15.3:0/1787738603 conn(0x7f7534102ae0 msgr2=0x7f7534104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 -- 172.21.15.3:0/1787738603 shutdown_connections 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.379+0000 7f7538a00640 1 -- 172.21.15.3:0/1787738603 wait complete. 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.380+0000 7f7538a00640 1 Processor -- start 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.381+0000 7f7538a00640 1 -- start start 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.381+0000 7f7538a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f753419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.381+0000 7f7538a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f753410b8d0 con 0x7f7534107370 2024-08-01T04:06:10.421 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.381+0000 7f7533400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f753419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.381+0000 7f7533400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f753419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50304/0 (socket says 172.21.15.3:50304) 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.381+0000 7f7533400640 1 -- 172.21.15.3:0/783205055 learned_addr learned my addr 172.21.15.3:0/783205055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.382+0000 7f7533400640 1 -- 172.21.15.3:0/783205055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753419b880 con 0x7f7534107370 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.382+0000 7f7533400640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f753419b340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f75240097f0 tx=0x7f7524030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.382+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7524037040 con 0x7f7534107370 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.382+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7524040070 con 0x7f7534107370 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.382+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f752403a390 con 0x7f7534107370 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.382+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753419bb10 con 0x7f7534107370 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.383+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753419e800 con 0x7f7534107370 2024-08-01T04:06:10.422 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.384+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f752403b430 con 0x7f7534107370 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.384+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75341077f0 con 0x7f7534107370 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.384+0000 7f752be00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f7518038f70 0x7f751803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.384+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f752406c0d0 con 0x7f7534107370 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.384+0000 7f7532a00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f7518038f70 0x7f751803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.385+0000 7f7532a00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f7518038f70 0x7f751803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098c0 tx=0x7f751c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.390+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f75240371e0 con 0x7f7534107370 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.584+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f753419bf90 con 0x7f7534107370 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.585+0000 7f752be00640 1 -- 172.21.15.3:0/783205055 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+452 (secure 0 0 0) 0x7f7524032dd0 con 0x7f7534107370 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.588+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f7518038f70 msgr2=0x7f751803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.588+0000 7f7538a00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f7518038f70 0x7f751803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098c0 tx=0x7f751c006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:10.423 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.588+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 msgr2=0x7f753419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.588+0000 7f7538a00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f753419b340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f75240097f0 tx=0x7f7524030d80 comp rx=0 tx=0).stop 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.589+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 shutdown_connections 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.589+0000 7f7538a00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f7518038f70 0x7f751803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.589+0000 7f7538a00640 1 --2- 172.21.15.3:0/783205055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534107370 0x7f753419b340 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.589+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 >> 172.21.15.3:0/783205055 conn(0x7f7534102ae0 msgr2=0x7f7534103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.589+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 shutdown_connections 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:09.589+0000 7f7538a00640 1 -- 172.21.15.3:0/783205055 wait complete. 2024-08-01T04:06:10.424 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2024-08-01T04:06:11.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:11 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/816254532' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-08-01T04:06:11.548 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ignoring --setuser ceph since I am not root 2024-08-01T04:06:11.548 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ignoring --setgroup ceph since I am not root 2024-08-01T04:06:11.809 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:11.550+0000 7f9f733da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:06:11.809 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:11.726+0000 7f9f733da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:06:11.809 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:11.807+0000 7f9f733da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:06:11.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.975+0000 7fb4fb200640 1 Processor -- start 2024-08-01T04:06:11.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.976+0000 7fb4fb200640 1 -- start start 2024-08-01T04:06:11.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.976+0000 7fb4fb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc107770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:11.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.976+0000 7fb4fb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc107d40 con 0x7fb4fc107370 2024-08-01T04:06:11.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.977+0000 7fb4f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:11.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.977+0000 7fb4f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50312/0 (socket says 172.21.15.3:50312) 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.977+0000 7fb4f9e00640 1 -- 172.21.15.3:0/1207576060 learned_addr learned my addr 172.21.15.3:0/1207576060 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.977+0000 7fb4f9e00640 1 -- 172.21.15.3:0/1207576060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc108520 con 0x7fb4fc107370 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.978+0000 7fb4f9e00640 1 --2- 172.21.15.3:0/1207576060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc107770 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009820 tx=0x7fb4ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c382adb2e2abe621 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.979+0000 7fb4f8a00640 1 -- 172.21.15.3:0/1207576060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec037070 con 0x7fb4fc107370 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.979+0000 7fb4f8a00640 1 -- 172.21.15.3:0/1207576060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4ec032900 con 0x7fb4fc107370 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.979+0000 7fb4f8a00640 1 -- 172.21.15.3:0/1207576060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec030850 con 0x7fb4fc107370 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.979+0000 7fb4fb200640 1 -- 172.21.15.3:0/1207576060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 msgr2=0x7fb4fc107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.979+0000 7fb4fb200640 1 --2- 172.21.15.3:0/1207576060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc107770 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009820 tx=0x7fb4ec02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.980+0000 7fb4fb200640 1 -- 172.21.15.3:0/1207576060 shutdown_connections 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.980+0000 7fb4fb200640 1 --2- 172.21.15.3:0/1207576060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc107770 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:11.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.980+0000 7fb4fb200640 1 -- 172.21.15.3:0/1207576060 >> 172.21.15.3:0/1207576060 conn(0x7fb4fc102ae0 msgr2=0x7fb4fc104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.980+0000 7fb4fb200640 1 -- 172.21.15.3:0/1207576060 shutdown_connections 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.980+0000 7fb4fb200640 1 -- 172.21.15.3:0/1207576060 wait complete. 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.981+0000 7fb4fb200640 1 Processor -- start 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.981+0000 7fb4fb200640 1 -- start start 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.981+0000 7fb4fb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.981+0000 7fb4fb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc10b8d0 con 0x7fb4fc107370 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.982+0000 7fb4f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.982+0000 7fb4f9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50324/0 (socket says 172.21.15.3:50324) 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.982+0000 7fb4f9e00640 1 -- 172.21.15.3:0/816254532 learned_addr learned my addr 172.21.15.3:0/816254532 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.982+0000 7fb4f9e00640 1 -- 172.21.15.3:0/816254532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc19da90 con 0x7fb4fc107370 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.982+0000 7fb4f9e00640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc19d550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0097f0 tx=0x7fb4ec030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:11.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.982+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec037040 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.983+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4ec040070 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.983+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc19dd20 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.983+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec03a390 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.983+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc1a0a10 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.984+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb4ec03b430 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.984+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4fc1077f0 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.985+0000 7fb4f2a00640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb4dc038f70 0x7fb4dc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.985+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4ec06c0d0 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.985+0000 7fb4f9400640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb4dc038f70 0x7fb4dc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.986+0000 7fb4f9400640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb4dc038f70 0x7fb4dc03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40098c0 tx=0x7fb4e4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:10.990+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb4ec0371e0 con 0x7fb4fc107370 2024-08-01T04:06:11.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.233+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb4fc19e1a0 con 0x7fb4fc107370 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.287+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb4ec032dd0 con 0x7fb4fc107370 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.290+0000 7fb4f2a00640 1 -- 172.21.15.3:0/816254532 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb4ec032900 con 0x7fb4fc107370 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.290+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb4dc038f70 msgr2=0x7fb4dc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.290+0000 7fb4fb200640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb4dc038f70 0x7fb4dc03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40098c0 tx=0x7fb4e4006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.290+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 msgr2=0x7fb4fc19d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.290+0000 7fb4fb200640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc19d550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0097f0 tx=0x7fb4ec030d80 comp rx=0 tx=0).stop 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.292+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 shutdown_connections 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.292+0000 7fb4fb200640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fb4dc038f70 0x7fb4dc03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.292+0000 7fb4fb200640 1 --2- 172.21.15.3:0/816254532 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4fc107370 0x7fb4fc19d550 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.292+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 >> 172.21.15.3:0/816254532 conn(0x7fb4fc102ae0 msgr2=0x7fb4fc103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.293+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 shutdown_connections 2024-08-01T04:06:11.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:11.293+0000 7fb4fb200640 1 -- 172.21.15.3:0/816254532 wait complete. 2024-08-01T04:06:12.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:12 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/816254532' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-01T04:06:12.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:12 smithi003 ceph-mon[32262]: mgrmap e5: x(active, since 9s) 2024-08-01T04:06:12.377 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:12 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:12.124+0000 7f9f733da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:06:12.377 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:12 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:12.196+0000 7f9f733da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:06:13.434 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:13 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1652414482' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-01T04:06:13.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-08-01T04:06:13.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-01T04:06:13.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-01T04:06:13.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-01T04:06:13.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.596+0000 7fcc89200640 1 Processor -- start 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.596+0000 7fcc89200640 1 -- start start 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.596+0000 7fcc89200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8410a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.596+0000 7fcc89200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8407bdc0 con 0x7fcc84107e70 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.597+0000 7fcc83400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8410a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.597+0000 7fcc83400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8410a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54330/0 (socket says 172.21.15.3:54330) 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.597+0000 7fcc83400640 1 -- 172.21.15.3:0/2323857782 learned_addr learned my addr 172.21.15.3:0/2323857782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.597+0000 7fcc83400640 1 -- 172.21.15.3:0/2323857782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8410a7a0 con 0x7fcc84107e70 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.598+0000 7fcc83400640 1 --2- 172.21.15.3:0/2323857782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8410a260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009e80 tx=0x7fcc7402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8cc7fb01079d77 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.599+0000 7fcc82a00640 1 -- 172.21.15.3:0/2323857782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74036070 con 0x7fcc84107e70 2024-08-01T04:06:13.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.599+0000 7fcc82a00640 1 -- 172.21.15.3:0/2323857782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc740318c0 con 0x7fcc84107e70 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.599+0000 7fcc82a00640 1 -- 172.21.15.3:0/2323857782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74007ab0 con 0x7fcc84107e70 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.599+0000 7fcc89200640 1 -- 172.21.15.3:0/2323857782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 msgr2=0x7fcc8410a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.599+0000 7fcc89200640 1 --2- 172.21.15.3:0/2323857782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8410a260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009e80 tx=0x7fcc7402a7a0 comp rx=0 tx=0).stop 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.600+0000 7fcc89200640 1 -- 172.21.15.3:0/2323857782 shutdown_connections 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.600+0000 7fcc89200640 1 --2- 172.21.15.3:0/2323857782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8410a260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.600+0000 7fcc89200640 1 -- 172.21.15.3:0/2323857782 >> 172.21.15.3:0/2323857782 conn(0x7fcc84101a30 msgr2=0x7fcc84103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.600+0000 7fcc89200640 1 -- 172.21.15.3:0/2323857782 shutdown_connections 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.600+0000 7fcc89200640 1 -- 172.21.15.3:0/2323857782 wait complete. 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.601+0000 7fcc89200640 1 Processor -- start 2024-08-01T04:06:13.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.601+0000 7fcc89200640 1 -- start start 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.601+0000 7fcc89200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.601+0000 7fcc89200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8410b720 con 0x7fcc84107e70 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.601+0000 7fcc83400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.602+0000 7fcc83400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54332/0 (socket says 172.21.15.3:54332) 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.602+0000 7fcc83400640 1 -- 172.21.15.3:0/1652414482 learned_addr learned my addr 172.21.15.3:0/1652414482 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.602+0000 7fcc83400640 1 -- 172.21.15.3:0/1652414482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8419db10 con 0x7fcc84107e70 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.602+0000 7fcc83400640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8419d5d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc7402acc0 tx=0x7fcc74031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.602+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc74036050 con 0x7fcc84107e70 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.603+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc74037040 con 0x7fcc84107e70 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.603+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8419dda0 con 0x7fcc84107e70 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.603+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc7403d920 con 0x7fcc84107e70 2024-08-01T04:06:13.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.603+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc841a0a90 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.604+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fcc74044020 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.604+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc841082d0 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.604+0000 7fcc80c00640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 0x7fcc6803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.604+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc74042020 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.604+0000 7fcc7aa00640 1 -- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 msgr2=0x7fcc6803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.605+0000 7fcc7aa00640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 0x7fcc6803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.610+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcc7404b020 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.806+0000 7fcc7aa00640 1 -- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 msgr2=0x7fcc6803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.806+0000 7fcc7aa00640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 0x7fcc6803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.854+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcc841a10a0 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.855+0000 7fcc80c00640 1 -- 172.21.15.3:0/1652414482 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcc740371f0 con 0x7fcc84107e70 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 msgr2=0x7fcc6803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:06:13.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 0x7fcc6803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 msgr2=0x7fcc8419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8419d5d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc7402acc0 tx=0x7fcc74031ca0 comp rx=0 tx=0).stop 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 shutdown_connections 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7fcc68038fc0 0x7fcc6803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 --2- 172.21.15.3:0/1652414482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc84107e70 0x7fcc8419d5d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 >> 172.21.15.3:0/1652414482 conn(0x7fcc84101a30 msgr2=0x7fcc84102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.857+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 shutdown_connections 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:12.858+0000 7fcc89200640 1 -- 172.21.15.3:0/1652414482 wait complete. 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-08-01T04:06:13.517 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2024-08-01T04:06:13.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:13.432+0000 7f9f733da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:06:13.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:13.582+0000 7f9f733da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:06:13.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:13.652+0000 7f9f733da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:06:14.069 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:13.913+0000 7f9f733da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:06:14.069 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:13.990+0000 7f9f733da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:06:14.548 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:14.076+0000 7f9f733da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:06:14.548 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:14.249+0000 7f9f733da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:06:15.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:14.788+0000 7f9f733da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:06:15.420 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:15.158+0000 7f9f733da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:06:15.420 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:15.238+0000 7f9f733da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:06:15.420 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:15.418+0000 7f9f733da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:06:16.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:15.790+0000 7f9f733da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:06:16.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:15.864+0000 7f9f733da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:06:16.359 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:16.090+0000 7f9f733da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:06:16.359 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:16.279+0000 7f9f733da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:06:16.359 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:16.357+0000 7f9f733da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:06:16.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:16.457+0000 7f9f733da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:06:16.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:16.528+0000 7f9f733da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:06:17.350 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:17.162+0000 7f9f733da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:06:17.350 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:17.348+0000 7f9f733da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:06:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: Active manager daemon x restarted 2024-08-01T04:06:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: Activating manager daemon x 2024-08-01T04:06:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: osdmap e2: 0 total, 0 up, 0 in 2024-08-01T04:06:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: mgrmap e6: x(active, starting, since 0.00468819s) 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: Manager daemon x is now available 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:06:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:17 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:06:18.704 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:18 smithi003 ceph-mon[32262]: Found migration_current of "None". Setting to last migration. 2024-08-01T04:06:18.704 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:18 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:06:18.704 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:18 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:06:18.704 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:18 smithi003 ceph-mon[32262]: mgrmap e7: x(active, since 1.01434s) 2024-08-01T04:06:19.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-08-01T04:06:19.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-01T04:06:19.141 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-01T04:06:19.142 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-08-01T04:06:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.067+0000 7f3f4e400640 1 Processor -- start 2024-08-01T04:06:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.067+0000 7f3f4e400640 1 -- start start 2024-08-01T04:06:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.068+0000 7f3f4e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.068+0000 7f3f4e400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4807c7f0 con 0x7f3f4807de00 2024-08-01T04:06:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.068+0000 7f3f4d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:19.143 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.068+0000 7f3f4d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54348/0 (socket says 172.21.15.3:54348) 2024-08-01T04:06:19.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.068+0000 7f3f4d000640 1 -- 172.21.15.3:0/4038752366 learned_addr learned my addr 172.21.15.3:0/4038752366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:19.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.069+0000 7f3f4d000640 1 -- 172.21.15.3:0/4038752366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4807c970 con 0x7f3f4807de00 2024-08-01T04:06:19.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.070+0000 7f3f4d000640 1 --2- 172.21.15.3:0/4038752366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4807c2b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009820 tx=0x7f3f3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b81b4cc53044d13 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:19.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.070+0000 7f3f47400640 1 -- 172.21.15.3:0/4038752366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c037070 con 0x7f3f4807de00 2024-08-01T04:06:19.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.070+0000 7f3f47400640 1 -- 172.21.15.3:0/4038752366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f3c032900 con 0x7f3f4807de00 2024-08-01T04:06:19.144 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.070+0000 7f3f47400640 1 -- 172.21.15.3:0/4038752366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c030850 con 0x7f3f4807de00 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.071+0000 7f3f4e400640 1 -- 172.21.15.3:0/4038752366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 msgr2=0x7f3f4807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.071+0000 7f3f4e400640 1 --2- 172.21.15.3:0/4038752366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4807c2b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009820 tx=0x7f3f3c02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.071+0000 7f3f4e400640 1 -- 172.21.15.3:0/4038752366 shutdown_connections 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.071+0000 7f3f4e400640 1 --2- 172.21.15.3:0/4038752366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4807c2b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.071+0000 7f3f4e400640 1 -- 172.21.15.3:0/4038752366 >> 172.21.15.3:0/4038752366 conn(0x7f3f48102ae0 msgr2=0x7f3f48104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.071+0000 7f3f4e400640 1 -- 172.21.15.3:0/4038752366 shutdown_connections 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.072+0000 7f3f4e400640 1 -- 172.21.15.3:0/4038752366 wait complete. 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.073+0000 7f3f4e400640 1 Processor -- start 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.073+0000 7f3f4e400640 1 -- start start 2024-08-01T04:06:19.151 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.074+0000 7f3f4e400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.074+0000 7f3f4e400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810d9c0 con 0x7f3f4807de00 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.074+0000 7f3f4d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.074+0000 7f3f4d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4819d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54358/0 (socket says 172.21.15.3:54358) 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.074+0000 7f3f4d000640 1 -- 172.21.15.3:0/1048284498 learned_addr learned my addr 172.21.15.3:0/1048284498 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.074+0000 7f3f4d000640 1 -- 172.21.15.3:0/1048284498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4819da30 con 0x7f3f4807de00 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.075+0000 7f3f4d000640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4819d4f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0097f0 tx=0x7f3f3c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.075+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c037040 con 0x7f3f4807de00 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.075+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f3c040070 con 0x7f3f4807de00 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.075+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c03a390 con 0x7f3f4807de00 2024-08-01T04:06:19.152 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.075+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4819dcc0 con 0x7f3f4807de00 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.075+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f481a09b0 con 0x7f3f4807de00 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.077+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f3f3c03b430 con 0x7f3f4807de00 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.077+0000 7f3f45600640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.077+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 --> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f3003bb90 con 0x7f3f30038fc0 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.077+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f3c06c100 con 0x7f3f4807de00 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.077+0000 7f3f47e00640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 msgr2=0x7f3f3003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.077+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.278+0000 7f3f47e00640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 msgr2=0x7f3f3003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.278+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.679+0000 7f3f47e00640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 msgr2=0x7f3f3003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:14.679+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-01T04:06:19.153 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:15.480+0000 7f3f47e00640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 msgr2=0x7f3f3003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:15.480+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:17.082+0000 7f3f47e00640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 msgr2=0x7f3f3003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1530825959 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:17.082+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:17.355+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f3f3c032900 con 0x7f3f4807de00 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:17.356+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 msgr2=0x7f3f3003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:17.356+0000 7f3f45600640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.359+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f3f3c06d710 con 0x7f3f4807de00 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.360+0000 7f3f45600640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f3f3003c8f0 0x7f3f3003ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.360+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f3c046e50 con 0x7f3f3003c8f0 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.362+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f3f3003c8f0 0x7f3f3003ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.366+0000 7f3f47e00640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f3f3003c8f0 0x7f3f3003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f34003980 tx=0x7f3f340092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:19.154 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.367+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3f3c046e50 con 0x7f3f3003c8f0 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.376+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3f4807c330 con 0x7f3f3003c8f0 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.376+0000 7f3f45600640 1 -- 172.21.15.3:0/1048284498 <== mgr.14116 v2:172.21.15.3:6800/2156560379 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f3f4807c330 con 0x7f3f3003c8f0 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f3f3003c8f0 msgr2=0x7f3f3003ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f3f3003c8f0 0x7f3f3003ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f34003980 tx=0x7f3f340092b0 comp rx=0 tx=0).stop 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 msgr2=0x7f3f4819d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4819d4f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0097f0 tx=0x7f3f3c030d80 comp rx=0 tx=0).stop 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 shutdown_connections 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f3f3003c8f0 0x7f3f3003ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:6800/1530825959,v1:172.21.15.3:6801/1530825959] conn(0x7f3f30038fc0 0x7f3f3003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 --2- 172.21.15.3:0/1048284498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f4807de00 0x7f3f4819d4f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.377+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 >> 172.21.15.3:0/1048284498 conn(0x7f3f48102ae0 msgr2=0x7f3f481036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.378+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 shutdown_connections 2024-08-01T04:06:19.155 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:18.378+0000 7f3f4e400640 1 -- 172.21.15.3:0/1048284498 wait complete. 2024-08-01T04:06:19.156 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2024-08-01T04:06:19.156 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2024-08-01T04:06:19.680 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:19 smithi003 ceph-mon[32262]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-01T04:06:19.681 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:19 smithi003 ceph-mon[32262]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-01T04:06:19.681 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:19 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:19.681 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:19 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:20.761 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:20.345+0000 7f9ecae00640 -1 log_channel(cephadm) log [ERR] : [01/Aug/2024:04:06:20] ENGINE Error in HTTPServer.serve 2024-08-01T04:06:20.762 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: Traceback (most recent call last): 2024-08-01T04:06:20.762 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:06:20.762 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._connections.run(self.expiration_interval) 2024-08-01T04:06:20.762 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:06:20.762 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._run(expiration_interval) 2024-08-01T04:06:20.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:06:20.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:06:20.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:06:20.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:06:20.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:06:20.763 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: s = self.context.wrap_socket( 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: return self.sslsocket_class._create( 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self.do_handshake() 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._sslobj.do_handshake() 2024-08-01T04:06:20.764 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:06:20.765 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:20.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.707+0000 7fe3b4a00640 1 Processor -- start 2024-08-01T04:06:20.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.707+0000 7fe3b4a00640 1 -- start start 2024-08-01T04:06:20.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.707+0000 7fe3b4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:20.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.707+0000 7fe3b4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b007c7f0 con 0x7fe3b007de00 2024-08-01T04:06:20.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.708+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.708+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54422/0 (socket says 172.21.15.3:54422) 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.708+0000 7fe3ae800640 1 -- 172.21.15.3:0/923139619 learned_addr learned my addr 172.21.15.3:0/923139619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.708+0000 7fe3ae800640 1 -- 172.21.15.3:0/923139619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b007c970 con 0x7fe3b007de00 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.709+0000 7fe3ae800640 1 --2- 172.21.15.3:0/923139619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b007c2b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009e80 tx=0x7fe3a002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274d2f2fdb7976a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.710+0000 7fe3ade00640 1 -- 172.21.15.3:0/923139619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3a0036070 con 0x7fe3b007de00 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.710+0000 7fe3ade00640 1 -- 172.21.15.3:0/923139619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3a00318c0 con 0x7fe3b007de00 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.710+0000 7fe3ade00640 1 -- 172.21.15.3:0/923139619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3a0007ab0 con 0x7fe3b007de00 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.710+0000 7fe3b4a00640 1 -- 172.21.15.3:0/923139619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 msgr2=0x7fe3b007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.710+0000 7fe3b4a00640 1 --2- 172.21.15.3:0/923139619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b007c2b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009e80 tx=0x7fe3a002a7a0 comp rx=0 tx=0).stop 2024-08-01T04:06:20.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.711+0000 7fe3b4a00640 1 -- 172.21.15.3:0/923139619 shutdown_connections 2024-08-01T04:06:20.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.711+0000 7fe3b4a00640 1 --2- 172.21.15.3:0/923139619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b007c2b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:20.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.711+0000 7fe3b4a00640 1 -- 172.21.15.3:0/923139619 >> 172.21.15.3:0/923139619 conn(0x7fe3b0102ae0 msgr2=0x7fe3b0104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:20.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.711+0000 7fe3b4a00640 1 -- 172.21.15.3:0/923139619 shutdown_connections 2024-08-01T04:06:20.786 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.711+0000 7fe3b4a00640 1 -- 172.21.15.3:0/923139619 wait complete. 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.712+0000 7fe3b4a00640 1 Processor -- start 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.712+0000 7fe3b4a00640 1 -- start start 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.712+0000 7fe3b4a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.712+0000 7fe3b4a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b010d9c0 con 0x7fe3b007de00 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.713+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.713+0000 7fe3ae800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b019d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54426/0 (socket says 172.21.15.3:54426) 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.713+0000 7fe3ae800640 1 -- 172.21.15.3:0/3628049073 learned_addr learned my addr 172.21.15.3:0/3628049073 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.713+0000 7fe3ae800640 1 -- 172.21.15.3:0/3628049073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b019da30 con 0x7fe3b007de00 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.713+0000 7fe3ae800640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b019d4f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002acc0 tx=0x7fe3a0031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.714+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3a0036050 con 0x7fe3b007de00 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.714+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3a0037040 con 0x7fe3b007de00 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.714+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3a003d850 con 0x7fe3b007de00 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.714+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b019dcc0 con 0x7fe3b007de00 2024-08-01T04:06:20.787 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.714+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b01a09b0 con 0x7fe3b007de00 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.715+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fe3a0044020 con 0x7fe3b007de00 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.715+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b007c330 con 0x7fe3b007de00 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.715+0000 7fe3a7e00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe394038e60 0x7fe39403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.715+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe3a0042020 con 0x7fe3b007de00 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.716+0000 7fe3a5e00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe394038e60 0x7fe39403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.717+0000 7fe3a5e00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe394038e60 0x7fe39403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3980098c0 tx=0x7fe398006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:20.788 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.721+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe3a004b020 con 0x7fe3b007de00 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.928+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fe3b0103db0 con 0x7fe394038e60 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.940+0000 7fe3a7e00640 1 -- 172.21.15.3:0/3628049073 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3b0103db0 con 0x7fe394038e60 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe394038e60 msgr2=0x7fe39403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe394038e60 0x7fe39403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3980098c0 tx=0x7fe398006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 msgr2=0x7fe3b019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b019d4f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002acc0 tx=0x7fe3a0031ca0 comp rx=0 tx=0).stop 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 shutdown_connections 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe394038e60 0x7fe39403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 --2- 172.21.15.3:0/3628049073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe3b007de00 0x7fe3b019d4f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:20.790 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.943+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 >> 172.21.15.3:0/3628049073 conn(0x7fe3b0102ae0 msgr2=0x7fe3b01036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:20.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.944+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 shutdown_connections 2024-08-01T04:06:20.791 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:19.944+0000 7fe3b4a00640 1 -- 172.21.15.3:0/3628049073 wait complete. 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: mgrmap e8: x(active, since 2s) 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:20] ENGINE Bus STARTING 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:20] ENGINE Serving on http://172.21.15.3:8765 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:20] ENGINE Serving on https://172.21.15.3:7150 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:20] ENGINE Bus STARTED 2024-08-01T04:06:21.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:20 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:20] ENGINE Error in HTTPServer.serve 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:06:21.050 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-08-01T04:06:21.051 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:06:22.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-01T04:06:22.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.365+0000 7fe847e00640 1 Processor -- start 2024-08-01T04:06:22.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.366+0000 7fe847e00640 1 -- start start 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.366+0000 7fe847e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe848109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.366+0000 7fe847e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe84810a280 con 0x7fe8481098b0 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.366+0000 7fe846a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe848109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.367+0000 7fe846a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe848109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54442/0 (socket says 172.21.15.3:54442) 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.367+0000 7fe846a00640 1 -- 172.21.15.3:0/304838376 learned_addr learned my addr 172.21.15.3:0/304838376 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.367+0000 7fe846a00640 1 -- 172.21.15.3:0/304838376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84810a740 con 0x7fe8481098b0 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.368+0000 7fe846a00640 1 --2- 172.21.15.3:0/304838376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe848109cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe838009820 tx=0x7fe83802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231fdef2764d30ed server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.368+0000 7fe846000640 1 -- 172.21.15.3:0/304838376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe838037070 con 0x7fe8481098b0 2024-08-01T04:06:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.368+0000 7fe846000640 1 -- 172.21.15.3:0/304838376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe838032900 con 0x7fe8481098b0 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.368+0000 7fe846000640 1 -- 172.21.15.3:0/304838376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe838030850 con 0x7fe8481098b0 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 -- 172.21.15.3:0/304838376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 msgr2=0x7fe848109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 --2- 172.21.15.3:0/304838376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe848109cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe838009820 tx=0x7fe83802a270 comp rx=0 tx=0).stop 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 -- 172.21.15.3:0/304838376 shutdown_connections 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 --2- 172.21.15.3:0/304838376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe848109cb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 -- 172.21.15.3:0/304838376 >> 172.21.15.3:0/304838376 conn(0x7fe84807ccb0 msgr2=0x7fe84807d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 -- 172.21.15.3:0/304838376 shutdown_connections 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.369+0000 7fe847e00640 1 -- 172.21.15.3:0/304838376 wait complete. 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.370+0000 7fe847e00640 1 Processor -- start 2024-08-01T04:06:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.370+0000 7fe847e00640 1 -- start start 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.371+0000 7fe847e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe84819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.371+0000 7fe847e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe84810daf0 con 0x7fe8481098b0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.371+0000 7fe846a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe84819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.371+0000 7fe846a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe84819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54444/0 (socket says 172.21.15.3:54444) 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.371+0000 7fe846a00640 1 -- 172.21.15.3:0/1104757477 learned_addr learned my addr 172.21.15.3:0/1104757477 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.371+0000 7fe846a00640 1 -- 172.21.15.3:0/1104757477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84819da40 con 0x7fe8481098b0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.372+0000 7fe846a00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe84819d500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8380097f0 tx=0x7fe838030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.372+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe838037040 con 0x7fe8481098b0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.372+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe838040070 con 0x7fe8481098b0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.372+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe83803a390 con 0x7fe8481098b0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.372+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84819dcd0 con 0x7fe8481098b0 2024-08-01T04:06:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.372+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8481a09c0 con 0x7fe8481098b0 2024-08-01T04:06:22.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.373+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe838032900 con 0x7fe8481098b0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.373+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe848109d30 con 0x7fe8481098b0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.374+0000 7fe83f400640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe82c038fc0 0x7fe82c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.374+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe83806c500 con 0x7fe8481098b0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.374+0000 7fe83fe00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe82c038fc0 0x7fe82c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.375+0000 7fe83fe00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe82c038fc0 0x7fe82c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8300098c0 tx=0x7fe830006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.379+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe83806be50 con 0x7fe8481098b0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.575+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe848107be0 con 0x7fe82c038fc0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.576+0000 7fe83f400640 1 -- 172.21.15.3:0/1104757477 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe848107be0 con 0x7fe82c038fc0 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe82c038fc0 msgr2=0x7fe82c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe82c038fc0 0x7fe82c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8300098c0 tx=0x7fe830006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 msgr2=0x7fe84819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:22.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe84819d500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8380097f0 tx=0x7fe838030d80 comp rx=0 tx=0).stop 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 shutdown_connections 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fe82c038fc0 0x7fe82c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 --2- 172.21.15.3:0/1104757477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8481098b0 0x7fe84819d500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.579+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 >> 172.21.15.3:0/1104757477 conn(0x7fe84807ccb0 msgr2=0x7fe8481074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.580+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 shutdown_connections 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:21.580+0000 7fe847e00640 1 -- 172.21.15.3:0/1104757477 wait complete. 2024-08-01T04:06:22.249 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2024-08-01T04:06:22.735 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:22 smithi003 ceph-mon[32262]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:23.770 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-mon[32262]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:23.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-mon[32262]: Generating ssh key... 2024-08-01T04:06:23.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: Generating public/private rsa key pair. 2024-08-01T04:06:23.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: Your identification has been saved in /tmp/tmp4xd3oy1m/key 2024-08-01T04:06:23.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: Your public key has been saved in /tmp/tmp4xd3oy1m/key.pub 2024-08-01T04:06:23.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: The key fingerprint is: 2024-08-01T04:06:23.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: SHA256:TS8UtqKeC+fWlUQJfNioRGpfcyZMX+ckKlFGL5XgTv8 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: The key's randomart image is: 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: +---[RSA 3072]----+ 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | ...+BBo+.o | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | ..o+*=B.= | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | o. .Bo% . . | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | . ..o &.+ | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | o S.+.o | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | . . o. . | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | . +. . E | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | +... | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: | .o | 2024-08-01T04:06:23.772 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: +----[SHA256]-----+ 2024-08-01T04:06:24.376 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.871+0000 7f0ae3e00640 1 Processor -- start 2024-08-01T04:06:24.377 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.871+0000 7f0ae3e00640 1 -- start start 2024-08-01T04:06:24.377 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.872+0000 7f0ae3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae4081640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:24.377 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.872+0000 7f0ae3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4081c10 con 0x7f0ae4081240 2024-08-01T04:06:24.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.872+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae4081640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:24.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.872+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae4081640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43408/0 (socket says 172.21.15.3:43408) 2024-08-01T04:06:24.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.872+0000 7f0ae2a00640 1 -- 172.21.15.3:0/2282017620 learned_addr learned my addr 172.21.15.3:0/2282017620 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:24.378 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.873+0000 7f0ae2a00640 1 -- 172.21.15.3:0/2282017620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae4081d90 con 0x7f0ae4081240 2024-08-01T04:06:24.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.874+0000 7f0ae2a00640 1 --2- 172.21.15.3:0/2282017620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae4081640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009e80 tx=0x7f0ad402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543420fe83f52209 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:24.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.874+0000 7f0ae1600640 1 -- 172.21.15.3:0/2282017620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ad4036070 con 0x7f0ae4081240 2024-08-01T04:06:24.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.874+0000 7f0ae1600640 1 -- 172.21.15.3:0/2282017620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ad40318c0 con 0x7f0ae4081240 2024-08-01T04:06:24.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.874+0000 7f0ae1600640 1 -- 172.21.15.3:0/2282017620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ad4007ab0 con 0x7f0ae4081240 2024-08-01T04:06:24.379 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 -- 172.21.15.3:0/2282017620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 msgr2=0x7f0ae4081640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:24.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 --2- 172.21.15.3:0/2282017620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae4081640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009e80 tx=0x7f0ad402a7a0 comp rx=0 tx=0).stop 2024-08-01T04:06:24.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 -- 172.21.15.3:0/2282017620 shutdown_connections 2024-08-01T04:06:24.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 --2- 172.21.15.3:0/2282017620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae4081640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:24.380 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 -- 172.21.15.3:0/2282017620 >> 172.21.15.3:0/2282017620 conn(0x7f0ae407cb50 msgr2=0x7f0ae407ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:24.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 -- 172.21.15.3:0/2282017620 shutdown_connections 2024-08-01T04:06:24.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.875+0000 7f0ae3e00640 1 -- 172.21.15.3:0/2282017620 wait complete. 2024-08-01T04:06:24.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.876+0000 7f0ae3e00640 1 Processor -- start 2024-08-01T04:06:24.381 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.877+0000 7f0ae3e00640 1 -- start start 2024-08-01T04:06:24.382 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.877+0000 7f0ae3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:24.382 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.877+0000 7f0ae3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4111ef0 con 0x7f0ae4081240 2024-08-01T04:06:24.382 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.877+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:24.382 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.877+0000 7f0ae2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae41a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43422/0 (socket says 172.21.15.3:43422) 2024-08-01T04:06:24.382 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.877+0000 7f0ae2a00640 1 -- 172.21.15.3:0/1033392694 learned_addr learned my addr 172.21.15.3:0/1033392694 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:24.383 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.878+0000 7f0ae2a00640 1 -- 172.21.15.3:0/1033392694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae41a1ec0 con 0x7f0ae4081240 2024-08-01T04:06:24.383 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.878+0000 7f0ae2a00640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae41a1980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0ad402acc0 tx=0x7f0ad4031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:24.383 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.878+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ad4036050 con 0x7f0ae4081240 2024-08-01T04:06:24.383 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.878+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ad4037040 con 0x7f0ae4081240 2024-08-01T04:06:24.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.878+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ad403d890 con 0x7f0ae4081240 2024-08-01T04:06:24.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.878+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae41a2150 con 0x7f0ae4081240 2024-08-01T04:06:24.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.879+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41a4e40 con 0x7f0ae4081240 2024-08-01T04:06:24.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.880+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0ad4044020 con 0x7f0ae4081240 2024-08-01T04:06:24.384 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.880+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae4112070 con 0x7f0ae4081240 2024-08-01T04:06:24.385 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.880+0000 7f0adb400640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f0ac8038f70 0x7f0ac803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:24.386 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.880+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0ad4042020 con 0x7f0ae4081240 2024-08-01T04:06:24.386 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.880+0000 7f0ae2000640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f0ac8038f70 0x7f0ac803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:24.386 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.881+0000 7f0ae2000640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f0ac8038f70 0x7f0ac803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00098c0 tx=0x7f0ad0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:24.386 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:22.886+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0ad404b020 con 0x7f0ae4081240 2024-08-01T04:06:24.386 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.080+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0ae407de60 con 0x7f0ac8038f70 2024-08-01T04:06:24.387 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.702+0000 7f0adb400640 1 -- 172.21.15.3:0/1033392694 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ae407de60 con 0x7f0ac8038f70 2024-08-01T04:06:24.387 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f0ac8038f70 msgr2=0x7f0ac803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:24.387 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f0ac8038f70 0x7f0ac803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00098c0 tx=0x7f0ad0006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:24.387 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 msgr2=0x7f0ae41a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:24.387 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae41a1980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0ad402acc0 tx=0x7f0ad4031ca0 comp rx=0 tx=0).stop 2024-08-01T04:06:24.387 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 shutdown_connections 2024-08-01T04:06:24.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f0ac8038f70 0x7f0ac803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:24.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 --2- 172.21.15.3:0/1033392694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ae4081240 0x7f0ae41a1980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:24.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 >> 172.21.15.3:0/1033392694 conn(0x7f0ae407cb50 msgr2=0x7f0ae407d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:24.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.705+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 shutdown_connections 2024-08-01T04:06:24.388 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:23.706+0000 7f0ae3e00640 1 -- 172.21.15.3:0/1033392694 wait complete. 2024-08-01T04:06:24.936 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:24 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:24.936 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:24 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:25.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:25 smithi003 ceph-mon[32262]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:06:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.935+0000 7f8ad3200640 1 Processor -- start 2024-08-01T04:06:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.935+0000 7f8ad3200640 1 -- start start 2024-08-01T04:06:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.935+0000 7f8ad3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc109c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.935+0000 7f8ad3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc10a230 con 0x7f8acc109860 2024-08-01T04:06:25.892 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.936+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.936+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43434/0 (socket says 172.21.15.3:43434) 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.936+0000 7f8ad1e00640 1 -- 172.21.15.3:0/3522344691 learned_addr learned my addr 172.21.15.3:0/3522344691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.936+0000 7f8ad1e00640 1 -- 172.21.15.3:0/3522344691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc10a740 con 0x7f8acc109860 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.937+0000 7f8ad1e00640 1 --2- 172.21.15.3:0/3522344691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc109c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009820 tx=0x7f8ac002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcb49e96398a9de server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.938+0000 7f8ad0a00640 1 -- 172.21.15.3:0/3522344691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac0037070 con 0x7f8acc109860 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.938+0000 7f8ad0a00640 1 -- 172.21.15.3:0/3522344691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8ac0032900 con 0x7f8acc109860 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.938+0000 7f8ad0a00640 1 -- 172.21.15.3:0/3522344691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac0030850 con 0x7f8acc109860 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.938+0000 7f8ad3200640 1 -- 172.21.15.3:0/3522344691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 msgr2=0x7f8acc109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.938+0000 7f8ad3200640 1 --2- 172.21.15.3:0/3522344691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc109c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009820 tx=0x7f8ac002a270 comp rx=0 tx=0).stop 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.939+0000 7f8ad3200640 1 -- 172.21.15.3:0/3522344691 shutdown_connections 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.939+0000 7f8ad3200640 1 --2- 172.21.15.3:0/3522344691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc109c60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:25.893 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.939+0000 7f8ad3200640 1 -- 172.21.15.3:0/3522344691 >> 172.21.15.3:0/3522344691 conn(0x7f8acc07cb20 msgr2=0x7f8acc07cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.939+0000 7f8ad3200640 1 -- 172.21.15.3:0/3522344691 shutdown_connections 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.939+0000 7f8ad3200640 1 -- 172.21.15.3:0/3522344691 wait complete. 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad3200640 1 Processor -- start 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad3200640 1 -- start start 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc10daf0 con 0x7f8acc109860 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43446/0 (socket says 172.21.15.3:43446) 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.940+0000 7f8ad1e00640 1 -- 172.21.15.3:0/1372648190 learned_addr learned my addr 172.21.15.3:0/1372648190 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.941+0000 7f8ad1e00640 1 -- 172.21.15.3:0/1372648190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc19da80 con 0x7f8acc109860 2024-08-01T04:06:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.941+0000 7f8ad1e00640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc19d540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0030460 tx=0x7f8ac0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.941+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac0042070 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.941+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8ac0032be0 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.941+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac0037040 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.941+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc19dd10 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.942+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc1a0a00 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.943+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8ac0045070 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.943+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc109c60 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.943+0000 7f8ac6a00640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f8ab4038fc0 0x7f8ab403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.943+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8ac006bed0 con 0x7f8acc109860 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.944+0000 7f8ad1400640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f8ab4038fc0 0x7f8ab403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.944+0000 7f8ad1400640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f8ab4038fc0 0x7f8ab403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80098c0 tx=0x7f8ab8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:25.895 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:24.949+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8ac0045360 con 0x7f8acc109860 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.143+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f8acc107c00 con 0x7f8ab4038fc0 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.144+0000 7f8ac6a00640 1 -- 172.21.15.3:0/1372648190 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f8acc107c00 con 0x7f8ab4038fc0 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.146+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f8ab4038fc0 msgr2=0x7f8ab403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.146+0000 7f8ad3200640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f8ab4038fc0 0x7f8ab403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80098c0 tx=0x7f8ab8006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.146+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 msgr2=0x7f8acc19d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.146+0000 7f8ad3200640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc19d540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0030460 tx=0x7f8ac0030d80 comp rx=0 tx=0).stop 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.147+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 shutdown_connections 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.147+0000 7f8ad3200640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f8ab4038fc0 0x7f8ab403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.147+0000 7f8ad3200640 1 --2- 172.21.15.3:0/1372648190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8acc109860 0x7f8acc19d540 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.147+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 >> 172.21.15.3:0/1372648190 conn(0x7f8acc07cb20 msgr2=0x7f8acc1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.147+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 shutdown_connections 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:25.147+0000 7f8ad3200640 1 -- 172.21.15.3:0/1372648190 wait complete. 2024-08-01T04:06:25.896 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-01T04:06:25.897 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2024-08-01T04:06:25.897 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2024-08-01T04:06:27.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:27 smithi003 ceph-mon[32262]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:28.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:28 smithi003 ceph-mon[32262]: Deploying cephadm binary to smithi003 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.446+0000 7f9e77e00640 1 Processor -- start 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.446+0000 7f9e77e00640 1 -- start start 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.446+0000 7f9e77e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e78107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.446+0000 7f9e77e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78107d00 con 0x7f9e78107330 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.447+0000 7f9e76a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e78107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.447+0000 7f9e76a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e78107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43460/0 (socket says 172.21.15.3:43460) 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.447+0000 7f9e76a00640 1 -- 172.21.15.3:0/3919757271 learned_addr learned my addr 172.21.15.3:0/3919757271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:32.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.447+0000 7f9e76a00640 1 -- 172.21.15.3:0/3919757271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e78108530 con 0x7f9e78107330 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.449+0000 7f9e76a00640 1 --2- 172.21.15.3:0/3919757271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e78107730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009820 tx=0x7f9e6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc59e3ad1428cd server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.449+0000 7f9e75600640 1 -- 172.21.15.3:0/3919757271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68037070 con 0x7f9e78107330 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.449+0000 7f9e75600640 1 -- 172.21.15.3:0/3919757271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e68032900 con 0x7f9e78107330 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.449+0000 7f9e75600640 1 -- 172.21.15.3:0/3919757271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68030850 con 0x7f9e78107330 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 -- 172.21.15.3:0/3919757271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 msgr2=0x7f9e78107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 --2- 172.21.15.3:0/3919757271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e78107730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009820 tx=0x7f9e6802a270 comp rx=0 tx=0).stop 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 -- 172.21.15.3:0/3919757271 shutdown_connections 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 --2- 172.21.15.3:0/3919757271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e78107730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 -- 172.21.15.3:0/3919757271 >> 172.21.15.3:0/3919757271 conn(0x7f9e78102ac0 msgr2=0x7f9e78104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 -- 172.21.15.3:0/3919757271 shutdown_connections 2024-08-01T04:06:32.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.450+0000 7f9e77e00640 1 -- 172.21.15.3:0/3919757271 wait complete. 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.451+0000 7f9e77e00640 1 Processor -- start 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.452+0000 7f9e77e00640 1 -- start start 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.452+0000 7f9e77e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e7819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.452+0000 7f9e77e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7810b8e0 con 0x7f9e78107330 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.452+0000 7f9e76a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e7819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.452+0000 7f9e76a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e7819b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43470/0 (socket says 172.21.15.3:43470) 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.452+0000 7f9e76a00640 1 -- 172.21.15.3:0/702232826 learned_addr learned my addr 172.21.15.3:0/702232826 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.453+0000 7f9e76a00640 1 -- 172.21.15.3:0/702232826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7819b830 con 0x7f9e78107330 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.453+0000 7f9e76a00640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e7819b2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009940 tx=0x7f9e68032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.453+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68042070 con 0x7f9e78107330 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.453+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e680308c0 con 0x7f9e78107330 2024-08-01T04:06:32.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.453+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68037040 con 0x7f9e78107330 2024-08-01T04:06:32.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.453+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7819bac0 con 0x7f9e78107330 2024-08-01T04:06:32.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.454+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7819e7b0 con 0x7f9e78107330 2024-08-01T04:06:32.993 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.455+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9e68045070 con 0x7f9e78107330 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.455+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7810ba60 con 0x7f9e78107330 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.455+0000 7f9e6f400640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f9e5c038fc0 0x7f9e5c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.455+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9e6806c090 con 0x7f9e78107330 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.455+0000 7f9e76000640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f9e5c038fc0 0x7f9e5c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.456+0000 7f9e76000640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f9e5c038fc0 0x7f9e5c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098c0 tx=0x7f9e64006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.460+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9e6804b170 con 0x7f9e78107330 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:26.659+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}) -- 0x7f9e78103d10 con 0x7f9e5c038fc0 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.249+0000 7f9e6f400640 1 -- 172.21.15.3:0/702232826 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f9e78103d10 con 0x7f9e5c038fc0 2024-08-01T04:06:32.994 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f9e5c038fc0 msgr2=0x7f9e5c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f9e5c038fc0 0x7f9e5c03b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098c0 tx=0x7f9e64006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 msgr2=0x7f9e7819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e7819b2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009940 tx=0x7f9e68032ce0 comp rx=0 tx=0).stop 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 shutdown_connections 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f9e5c038fc0 0x7f9e5c03b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 --2- 172.21.15.3:0/702232826 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e78107330 0x7f9e7819b2f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.253+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 >> 172.21.15.3:0/702232826 conn(0x7f9e78102ac0 msgr2=0x7f9e78103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.254+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 shutdown_connections 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:32.254+0000 7f9e77e00640 1 -- 172.21.15.3:0/702232826 wait complete. 2024-08-01T04:06:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2024-08-01T04:06:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:33 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:33.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:33 smithi003 ceph-mon[32262]: Added host smithi003 2024-08-01T04:06:33.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:33 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.615+0000 7f4642800640 1 Processor -- start 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.615+0000 7f4642800640 1 -- start start 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.616+0000 7f4642800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c109c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.616+0000 7f4642800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10a230 con 0x7f463c109860 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.617+0000 7f463b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.617+0000 7f463b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37934/0 (socket says 172.21.15.3:37934) 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.617+0000 7f463b400640 1 -- 172.21.15.3:0/2062548620 learned_addr learned my addr 172.21.15.3:0/2062548620 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:34.769 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.617+0000 7f463b400640 1 -- 172.21.15.3:0/2062548620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c10a920 con 0x7f463c109860 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.618+0000 7f463b400640 1 --2- 172.21.15.3:0/2062548620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c109c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f462c009820 tx=0x7f462c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48c9c6e84b4796b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.618+0000 7f463aa00640 1 -- 172.21.15.3:0/2062548620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f462c037070 con 0x7f463c109860 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.618+0000 7f463aa00640 1 -- 172.21.15.3:0/2062548620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f462c032900 con 0x7f463c109860 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.618+0000 7f463aa00640 1 -- 172.21.15.3:0/2062548620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f462c030850 con 0x7f463c109860 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.619+0000 7f4642800640 1 -- 172.21.15.3:0/2062548620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 msgr2=0x7f463c109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.619+0000 7f4642800640 1 --2- 172.21.15.3:0/2062548620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c109c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f462c009820 tx=0x7f462c02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.620+0000 7f4642800640 1 -- 172.21.15.3:0/2062548620 shutdown_connections 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.620+0000 7f4642800640 1 --2- 172.21.15.3:0/2062548620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c109c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:34.770 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.620+0000 7f4642800640 1 -- 172.21.15.3:0/2062548620 >> 172.21.15.3:0/2062548620 conn(0x7f463c07cd60 msgr2=0x7f463c07d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.620+0000 7f4642800640 1 -- 172.21.15.3:0/2062548620 shutdown_connections 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.620+0000 7f4642800640 1 -- 172.21.15.3:0/2062548620 wait complete. 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.621+0000 7f4642800640 1 Processor -- start 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.621+0000 7f4642800640 1 -- start start 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.622+0000 7f4642800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.622+0000 7f4642800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f463c10dcd0 con 0x7f463c109860 2024-08-01T04:06:34.774 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.622+0000 7f463b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.622+0000 7f463b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37938/0 (socket says 172.21.15.3:37938) 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.622+0000 7f463b400640 1 -- 172.21.15.3:0/3385102818 learned_addr learned my addr 172.21.15.3:0/3385102818 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f463b400640 1 -- 172.21.15.3:0/3385102818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c19dc60 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f463b400640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c19d720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f462c009940 tx=0x7f462c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f462c042070 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f462c032be0 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f462c037040 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c19def0 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.623+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a0be0 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.624+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f462c045070 con 0x7f463c109860 2024-08-01T04:06:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.625+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c109c60 con 0x7f463c109860 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.625+0000 7f4638c00640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f4620038f70 0x7f462003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.625+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f462c06bfb0 con 0x7f463c109860 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.625+0000 7f4633e00640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f4620038f70 0x7f462003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.626+0000 7f4633e00640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f4620038f70 0x7f462003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f46240098c0 tx=0x7f4624006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.632+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f462c06b900 con 0x7f463c109860 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.846+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f463c107b80 con 0x7f4620038f70 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.855+0000 7f4638c00640 1 -- 172.21.15.3:0/3385102818 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f463c107b80 con 0x7f4620038f70 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f4620038f70 msgr2=0x7f462003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f4620038f70 0x7f462003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f46240098c0 tx=0x7f4624006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 msgr2=0x7f463c19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c19d720 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f462c009940 tx=0x7f462c0326e0 comp rx=0 tx=0).stop 2024-08-01T04:06:34.776 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 shutdown_connections 2024-08-01T04:06:34.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f4620038f70 0x7f462003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:34.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 --2- 172.21.15.3:0/3385102818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f463c109860 0x7f463c19d720 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:34.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 >> 172.21.15.3:0/3385102818 conn(0x7f463c07cd60 msgr2=0x7f463c107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:34.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.858+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 shutdown_connections 2024-08-01T04:06:34.777 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:33.859+0000 7f4642800640 1 -- 172.21.15.3:0/3385102818 wait complete. 2024-08-01T04:06:34.777 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2024-08-01T04:06:35.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:34 smithi003 ceph-mon[32262]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:35.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:34 smithi003 ceph-mon[32262]: Saving service mon spec with placement count:5 2024-08-01T04:06:35.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:34 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:36.940 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:36 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:36.941 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:36 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:36.941 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:36 smithi003 ceph-mon[32262]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:36.941 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:36 smithi003 ceph-mon[32262]: Saving service mgr spec with placement count:2 2024-08-01T04:06:36.941 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:36 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:36.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-01T04:06:36.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.030+0000 7fbb45200640 1 Processor -- start 2024-08-01T04:06:36.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.030+0000 7fbb45200640 1 -- start start 2024-08-01T04:06:36.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.030+0000 7fbb45200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb40107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:36.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.030+0000 7fbb45200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40107f00 con 0x7fbb40107530 2024-08-01T04:06:36.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.031+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb40107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:36.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.031+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb40107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37952/0 (socket says 172.21.15.3:37952) 2024-08-01T04:06:36.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.031+0000 7fbb3de00640 1 -- 172.21.15.3:0/988323163 learned_addr learned my addr 172.21.15.3:0/988323163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:36.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.031+0000 7fbb3de00640 1 -- 172.21.15.3:0/988323163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb40108730 con 0x7fbb40107530 2024-08-01T04:06:36.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.033+0000 7fbb3de00640 1 --2- 172.21.15.3:0/988323163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb40107930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009820 tx=0x7fbb3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c72ae562889c70 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:36.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.033+0000 7fbb3ca00640 1 -- 172.21.15.3:0/988323163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb30037070 con 0x7fbb40107530 2024-08-01T04:06:36.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.033+0000 7fbb3ca00640 1 -- 172.21.15.3:0/988323163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb30032920 con 0x7fbb40107530 2024-08-01T04:06:36.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.033+0000 7fbb3ca00640 1 -- 172.21.15.3:0/988323163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb30030810 con 0x7fbb40107530 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 -- 172.21.15.3:0/988323163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 msgr2=0x7fbb40107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 --2- 172.21.15.3:0/988323163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb40107930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009820 tx=0x7fbb3002a270 comp rx=0 tx=0).stop 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 -- 172.21.15.3:0/988323163 shutdown_connections 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 --2- 172.21.15.3:0/988323163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb40107930 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 -- 172.21.15.3:0/988323163 >> 172.21.15.3:0/988323163 conn(0x7fbb40102d00 msgr2=0x7fbb40105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 -- 172.21.15.3:0/988323163 shutdown_connections 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.034+0000 7fbb45200640 1 -- 172.21.15.3:0/988323163 wait complete. 2024-08-01T04:06:36.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.036+0000 7fbb45200640 1 Processor -- start 2024-08-01T04:06:36.957 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.036+0000 7fbb45200640 1 -- start start 2024-08-01T04:06:36.958 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.036+0000 7fbb45200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb4007db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:36.958 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.036+0000 7fbb45200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4010bae0 con 0x7fbb40107530 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.037+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb4007db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.037+0000 7fbb3de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb4007db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37964/0 (socket says 172.21.15.3:37964) 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.037+0000 7fbb3de00640 1 -- 172.21.15.3:0/335936339 learned_addr learned my addr 172.21.15.3:0/335936339 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.037+0000 7fbb3de00640 1 -- 172.21.15.3:0/335936339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4007e0a0 con 0x7fbb40107530 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.038+0000 7fbb3de00640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb4007db60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb300097f0 tx=0x7fbb300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.038+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb30037040 con 0x7fbb40107530 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.038+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4007c290 con 0x7fbb40107530 2024-08-01T04:06:36.959 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.038+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb30030cc0 con 0x7fbb40107530 2024-08-01T04:06:36.960 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.038+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3003a640 con 0x7fbb40107530 2024-08-01T04:06:36.960 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.038+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4007c7d0 con 0x7fbb40107530 2024-08-01T04:06:36.960 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.039+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbb3003a8f0 con 0x7fbb40107530 2024-08-01T04:06:36.962 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.039+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb401079b0 con 0x7fbb40107530 2024-08-01T04:06:36.962 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.040+0000 7fbb36a00640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fbb24038fc0 0x7fbb2403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:36.962 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.040+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb3003ba40 con 0x7fbb40107530 2024-08-01T04:06:36.962 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.040+0000 7fbb3d400640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fbb24038fc0 0x7fbb2403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:36.962 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.041+0000 7fbb3d400640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fbb24038fc0 0x7fbb2403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098c0 tx=0x7fbb2c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:36.962 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.045+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbb300328c0 con 0x7fbb40107530 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.261+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbb4007d150 con 0x7fbb24038fc0 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.266+0000 7fbb36a00640 1 -- 172.21.15.3:0/335936339 <== mgr.14116 v2:172.21.15.3:6800/2156560379 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbb4007d150 con 0x7fbb24038fc0 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fbb24038fc0 msgr2=0x7fbb2403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fbb24038fc0 0x7fbb2403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098c0 tx=0x7fbb2c006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 msgr2=0x7fbb4007db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb4007db60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb300097f0 tx=0x7fbb300327b0 comp rx=0 tx=0).stop 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 shutdown_connections 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fbb24038fc0 0x7fbb2403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 --2- 172.21.15.3:0/335936339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb40107530 0x7fbb4007db60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 >> 172.21.15.3:0/335936339 conn(0x7fbb40102d00 msgr2=0x7fbb401038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.269+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 shutdown_connections 2024-08-01T04:06:36.963 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:36.270+0000 7fbb45200640 1 -- 172.21.15.3:0/335936339 wait complete. 2024-08-01T04:06:38.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:38 smithi003 ceph-mon[32262]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:06:38.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:38 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2117508657' entity='client.admin' 2024-08-01T04:06:38.460 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.515+0000 7fc3d8a00640 1 Processor -- start 2024-08-01T04:06:38.460 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.516+0000 7fc3d8a00640 1 -- start start 2024-08-01T04:06:38.460 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.516+0000 7fc3d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d4109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.516+0000 7fc3d8a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410a1f0 con 0x7fc3d41098b0 2024-08-01T04:06:38.465 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.516+0000 7fc3d1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d4109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.517+0000 7fc3d1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d4109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37968/0 (socket says 172.21.15.3:37968) 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.517+0000 7fc3d1e00640 1 -- 172.21.15.3:0/2662301251 learned_addr learned my addr 172.21.15.3:0/2662301251 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.517+0000 7fc3d1e00640 1 -- 172.21.15.3:0/2662301251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d410a370 con 0x7fc3d41098b0 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.518+0000 7fc3d1e00640 1 --2- 172.21.15.3:0/2662301251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d4109cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009820 tx=0x7fc3c402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa29404d6c0ce54 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.518+0000 7fc3d0a00640 1 -- 172.21.15.3:0/2662301251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4037070 con 0x7fc3d41098b0 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.518+0000 7fc3d0a00640 1 -- 172.21.15.3:0/2662301251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4032920 con 0x7fc3d41098b0 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.519+0000 7fc3d0a00640 1 -- 172.21.15.3:0/2662301251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4030890 con 0x7fc3d41098b0 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.519+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2662301251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 msgr2=0x7fc3d4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.519+0000 7fc3d8a00640 1 --2- 172.21.15.3:0/2662301251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d4109cb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009820 tx=0x7fc3c402a300 comp rx=0 tx=0).stop 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.519+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2662301251 shutdown_connections 2024-08-01T04:06:38.466 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.519+0000 7fc3d8a00640 1 --2- 172.21.15.3:0/2662301251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d4109cb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.519+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2662301251 >> 172.21.15.3:0/2662301251 conn(0x7fc3d407cd60 msgr2=0x7fc3d407d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.520+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2662301251 shutdown_connections 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.520+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2662301251 wait complete. 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d8a00640 1 Processor -- start 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d8a00640 1 -- start start 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d8a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410db80 con 0x7fc3d41098b0 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37974/0 (socket says 172.21.15.3:37974) 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.521+0000 7fc3d1e00640 1 -- 172.21.15.3:0/2117508657 learned_addr learned my addr 172.21.15.3:0/2117508657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.522+0000 7fc3d1e00640 1 -- 172.21.15.3:0/2117508657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d419dc40 con 0x7fc3d41098b0 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.522+0000 7fc3d1e00640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d419d700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40097f0 tx=0x7fc3c40308b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:38.467 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.522+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4037040 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.522+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4040070 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.522+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d419ded0 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.522+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c403a390 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.523+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d41a0bc0 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.524+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc3c403b430 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.524+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d4109d30 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.524+0000 7fc3caa00640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc3b8038f70 0x7fc3b803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.524+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc3c406c1d0 con 0x7fc3d41098b0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.525+0000 7fc3d1400640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc3b8038f70 0x7fc3b803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.525+0000 7fc3d1400640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc3b8038f70 0x7fc3b803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00098c0 tx=0x7fc3c0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:38.468 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.529+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc3c4030de0 con 0x7fc3d41098b0 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.726+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc3d41a11b0 con 0x7fc3d41098b0 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.731+0000 7fc3caa00640 1 -- 172.21.15.3:0/2117508657 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fc3c406bb20 con 0x7fc3d41098b0 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.733+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc3b8038f70 msgr2=0x7fc3b803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.733+0000 7fc3d8a00640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc3b8038f70 0x7fc3b803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00098c0 tx=0x7fc3c0006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.733+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 msgr2=0x7fc3d419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.733+0000 7fc3d8a00640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d419d700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40097f0 tx=0x7fc3c40308b0 comp rx=0 tx=0).stop 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.734+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 shutdown_connections 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.734+0000 7fc3d8a00640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc3b8038f70 0x7fc3b803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.734+0000 7fc3d8a00640 1 --2- 172.21.15.3:0/2117508657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc3d41098b0 0x7fc3d419d700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.734+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 >> 172.21.15.3:0/2117508657 conn(0x7fc3d407cd60 msgr2=0x7fc3d4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.734+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 shutdown_connections 2024-08-01T04:06:38.469 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:37.734+0000 7fc3d8a00640 1 -- 172.21.15.3:0/2117508657 wait complete. 2024-08-01T04:06:40.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:40 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:40.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:40 smithi003 ceph-mon[32262]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:06:40.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:40 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2478065705' entity='client.admin' 2024-08-01T04:06:40.204 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.276+0000 7fc7a8600640 1 Processor -- start 2024-08-01T04:06:40.204 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.276+0000 7fc7a8600640 1 -- start start 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.277+0000 7fc7a8600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a0107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.277+0000 7fc7a8600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a0107ed0 con 0x7fc7a0107500 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.277+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a0107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.277+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a0107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37990/0 (socket says 172.21.15.3:37990) 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.277+0000 7fc7a5c00640 1 -- 172.21.15.3:0/487864937 learned_addr learned my addr 172.21.15.3:0/487864937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.278+0000 7fc7a5c00640 1 -- 172.21.15.3:0/487864937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a0108700 con 0x7fc7a0107500 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.279+0000 7fc7a5c00640 1 --2- 172.21.15.3:0/487864937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a0107900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc790009e80 tx=0x7fc79002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bf1803fe896040 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.279+0000 7fc79fe00640 1 -- 172.21.15.3:0/487864937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc790036070 con 0x7fc7a0107500 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.279+0000 7fc79fe00640 1 -- 172.21.15.3:0/487864937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7900318e0 con 0x7fc7a0107500 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.279+0000 7fc79fe00640 1 -- 172.21.15.3:0/487864937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc790007a00 con 0x7fc7a0107500 2024-08-01T04:06:40.205 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 -- 172.21.15.3:0/487864937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 msgr2=0x7fc7a0107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 --2- 172.21.15.3:0/487864937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a0107900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc790009e80 tx=0x7fc79002a7a0 comp rx=0 tx=0).stop 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 -- 172.21.15.3:0/487864937 shutdown_connections 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 --2- 172.21.15.3:0/487864937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a0107900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 -- 172.21.15.3:0/487864937 >> 172.21.15.3:0/487864937 conn(0x7fc7a0102cb0 msgr2=0x7fc7a01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 -- 172.21.15.3:0/487864937 shutdown_connections 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.280+0000 7fc7a8600640 1 -- 172.21.15.3:0/487864937 wait complete. 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.282+0000 7fc7a8600640 1 Processor -- start 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.282+0000 7fc7a8600640 1 -- start start 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.283+0000 7fc7a8600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.283+0000 7fc7a8600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a010bab0 con 0x7fc7a0107500 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.283+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:40.206 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.283+0000 7fc7a5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a019b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37996/0 (socket says 172.21.15.3:37996) 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.283+0000 7fc7a5c00640 1 -- 172.21.15.3:0/2478065705 learned_addr learned my addr 172.21.15.3:0/2478065705 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.283+0000 7fc7a5c00640 1 -- 172.21.15.3:0/2478065705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a019ba70 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.284+0000 7fc7a5c00640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a019b530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc79002acc0 tx=0x7fc790031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.284+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc790036050 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.284+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc790037040 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.284+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc79003b890 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.284+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a019bd00 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.284+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a019e9f0 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.285+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc790044020 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.286+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a010bc30 con 0x7fc7a0107500 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.286+0000 7fc79e000640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc784039020 0x7fc78403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:40.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.286+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc790042020 con 0x7fc7a0107500 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.286+0000 7fc7a5200640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc784039020 0x7fc78403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.287+0000 7fc7a5200640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc784039020 0x7fc78403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098c0 tx=0x7fc78c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.292+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc79004b020 con 0x7fc7a0107500 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.487+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc7a0107980 con 0x7fc7a0107500 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.492+0000 7fc79e000640 1 -- 172.21.15.3:0/2478065705 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc790039d90 con 0x7fc7a0107500 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc784039020 msgr2=0x7fc78403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc784039020 0x7fc78403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098c0 tx=0x7fc78c006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 msgr2=0x7fc7a019b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a019b530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc79002acc0 tx=0x7fc790031cc0 comp rx=0 tx=0).stop 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 shutdown_connections 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fc784039020 0x7fc78403b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 --2- 172.21.15.3:0/2478065705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7a0107500 0x7fc7a019b530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.495+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 >> 172.21.15.3:0/2478065705 conn(0x7fc7a0102cb0 msgr2=0x7fc7a0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:40.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.496+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 shutdown_connections 2024-08-01T04:06:40.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:39.496+0000 7fc7a8600640 1 -- 172.21.15.3:0/2478065705 wait complete. 2024-08-01T04:06:40.209 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2024-08-01T04:06:42.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:41 smithi003 ceph-mon[32262]: from='mgr.14116 172.21.15.3:0/2974156505' entity='mgr.x' 2024-08-01T04:06:42.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:41 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2497785734' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-08-01T04:06:42.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:41 smithi003 ceph-mon[32262]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:06:42.049 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:41 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ignoring --setuser ceph since I am not root 2024-08-01T04:06:42.049 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:41 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ignoring --setgroup ceph since I am not root 2024-08-01T04:06:42.366 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:42.087+0000 7f3b13840040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:06:42.366 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:42.281+0000 7f3b13840040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:06:42.673 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:42.366+0000 7f3b13840040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:06:42.674 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:42.671+0000 7f3b13840040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:06:42.960 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:42.744+0000 7f3b13840040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:06:42.960 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2497785734' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-01T04:06:42.960 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:42 smithi003 ceph-mon[32262]: mgrmap e9: x(active, since 24s) 2024-08-01T04:06:42.977 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.766+0000 7f23c8400640 1 Processor -- start 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.766+0000 7f23c8400640 1 -- start start 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.766+0000 7f23c8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c0107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.766+0000 7f23c8400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0107f00 con 0x7f23c0107530 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.767+0000 7f23c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c0107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.767+0000 7f23c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c0107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38006/0 (socket says 172.21.15.3:38006) 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.767+0000 7f23c5a00640 1 -- 172.21.15.3:0/3644051264 learned_addr learned my addr 172.21.15.3:0/3644051264 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.768+0000 7f23c5a00640 1 -- 172.21.15.3:0/3644051264 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c0108730 con 0x7f23c0107530 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.769+0000 7f23c5a00640 1 --2- 172.21.15.3:0/3644051264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c0107930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009820 tx=0x7f23b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ebec97a845a764 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.769+0000 7f23bfe00640 1 -- 172.21.15.3:0/3644051264 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23b4037070 con 0x7f23c0107530 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.769+0000 7f23bfe00640 1 -- 172.21.15.3:0/3644051264 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4032920 con 0x7f23c0107530 2024-08-01T04:06:42.978 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.769+0000 7f23bfe00640 1 -- 172.21.15.3:0/3644051264 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23b4030810 con 0x7f23c0107530 2024-08-01T04:06:42.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 -- 172.21.15.3:0/3644051264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 msgr2=0x7f23c0107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:42.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 --2- 172.21.15.3:0/3644051264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c0107930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009820 tx=0x7f23b402a270 comp rx=0 tx=0).stop 2024-08-01T04:06:42.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 -- 172.21.15.3:0/3644051264 shutdown_connections 2024-08-01T04:06:42.979 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 --2- 172.21.15.3:0/3644051264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c0107930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:42.981 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 -- 172.21.15.3:0/3644051264 >> 172.21.15.3:0/3644051264 conn(0x7f23c0102d00 msgr2=0x7f23c0105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:42.981 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 -- 172.21.15.3:0/3644051264 shutdown_connections 2024-08-01T04:06:42.981 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.770+0000 7f23c8400640 1 -- 172.21.15.3:0/3644051264 wait complete. 2024-08-01T04:06:42.981 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.771+0000 7f23c8400640 1 Processor -- start 2024-08-01T04:06:42.981 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.772+0000 7f23c8400640 1 -- start start 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.772+0000 7f23c8400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.772+0000 7f23c8400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010bae0 con 0x7f23c0107530 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.772+0000 7f23c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.772+0000 7f23c5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38008/0 (socket says 172.21.15.3:38008) 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.772+0000 7f23c5a00640 1 -- 172.21.15.3:0/2497785734 learned_addr learned my addr 172.21.15.3:0/2497785734 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.773+0000 7f23c5a00640 1 -- 172.21.15.3:0/2497785734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c019ba60 con 0x7f23c0107530 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.773+0000 7f23c5a00640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c019b520 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23b40097f0 tx=0x7f23b40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.773+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23b4037040 con 0x7f23c0107530 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.773+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4030cc0 con 0x7f23c0107530 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.773+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c019bcf0 con 0x7f23c0107530 2024-08-01T04:06:42.982 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.773+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23b403a640 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.774+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c019e9e0 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.775+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f23b403a940 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.775+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c010bc60 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.775+0000 7f23be000640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f23a8038f70 0x7f23a803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.775+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f23b406c570 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.776+0000 7f23c5000640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f23a8038f70 0x7f23a803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.776+0000 7f23c5000640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f23a8038f70 0x7f23a803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0098c0 tx=0x7f23ac006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:40.781+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f23b403ba40 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.027+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f23c019ebb0 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.812+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f23b403ad30 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.814+0000 7f23be000640 1 -- 172.21.15.3:0/2497785734 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f23b402fd60 con 0x7f23c0107530 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.817+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f23a8038f70 msgr2=0x7f23a803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:42.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.817+0000 7f23c8400640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f23a8038f70 0x7f23a803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0098c0 tx=0x7f23ac006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.817+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 msgr2=0x7f23c019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.817+0000 7f23c8400640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c019b520 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23b40097f0 tx=0x7f23b40327b0 comp rx=0 tx=0).stop 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.818+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 shutdown_connections 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.818+0000 7f23c8400640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f23a8038f70 0x7f23a803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.818+0000 7f23c8400640 1 --2- 172.21.15.3:0/2497785734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23c0107530 0x7f23c019b520 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.818+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 >> 172.21.15.3:0/2497785734 conn(0x7f23c0102d00 msgr2=0x7f23c01038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.818+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 shutdown_connections 2024-08-01T04:06:42.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:41.818+0000 7f23c8400640 1 -- 172.21.15.3:0/2497785734 wait complete. 2024-08-01T04:06:44.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:43 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/4282260291' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-01T04:06:44.442 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.060+0000 7f3b13840040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:06:44.442 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.205+0000 7f3b13840040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:06:44.442 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.276+0000 7f3b13840040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:06:44.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-08-01T04:06:44.470 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.512+0000 7f201de00640 1 Processor -- start 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.513+0000 7f201de00640 1 -- start start 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.513+0000 7f201de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 0x7f2018106000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.513+0000 7f201de00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f201807be20 con 0x7f2018105c00 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.513+0000 7f201ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 0x7f2018106000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.514+0000 7f201ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 0x7f2018106000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49510/0 (socket says 172.21.15.3:49510) 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.514+0000 7f201ca00640 1 -- 172.21.15.3:0/4213283726 learned_addr learned my addr 172.21.15.3:0/4213283726 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.514+0000 7f201ca00640 1 -- 172.21.15.3:0/4213283726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2018106540 con 0x7f2018105c00 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.515+0000 7f201ca00640 1 --2- 172.21.15.3:0/4213283726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 0x7f2018106000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f200c009820 tx=0x7f200c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a445da3f3e151b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.515+0000 7f2017400640 1 -- 172.21.15.3:0/4213283726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f200c037070 con 0x7f2018105c00 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.515+0000 7f2017400640 1 -- 172.21.15.3:0/4213283726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c032920 con 0x7f2018105c00 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.515+0000 7f2017400640 1 -- 172.21.15.3:0/4213283726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f200c030810 con 0x7f2018105c00 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.516+0000 7f201de00640 1 -- 172.21.15.3:0/4213283726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 msgr2=0x7f2018106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.516+0000 7f201de00640 1 --2- 172.21.15.3:0/4213283726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 0x7f2018106000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f200c009820 tx=0x7f200c02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.516+0000 7f201de00640 1 -- 172.21.15.3:0/4213283726 shutdown_connections 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.516+0000 7f201de00640 1 --2- 172.21.15.3:0/4213283726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2018105c00 0x7f2018106000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.516+0000 7f201de00640 1 -- 172.21.15.3:0/4213283726 >> 172.21.15.3:0/4213283726 conn(0x7f2018101a30 msgr2=0x7f2018103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.517+0000 7f201de00640 1 -- 172.21.15.3:0/4213283726 shutdown_connections 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.517+0000 7f201de00640 1 -- 172.21.15.3:0/4213283726 wait complete. 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.518+0000 7f201de00640 1 Processor -- start 2024-08-01T04:06:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.518+0000 7f201de00640 1 -- start start 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.518+0000 7f201de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 0x7f201819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.518+0000 7f201de00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2018107680 con 0x7f201819d720 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.519+0000 7f201ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 0x7f201819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.519+0000 7f201ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 0x7f201819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49516/0 (socket says 172.21.15.3:49516) 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.519+0000 7f201ca00640 1 -- 172.21.15.3:0/4282260291 learned_addr learned my addr 172.21.15.3:0/4282260291 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.519+0000 7f201ca00640 1 -- 172.21.15.3:0/4282260291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201819e080 con 0x7f201819d720 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.520+0000 7f201ca00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 0x7f201819db40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f200c02a860 tx=0x7f200c030d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.520+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f200c037040 con 0x7f201819d720 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.520+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c042070 con 0x7f201819d720 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.520+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f200c03b3f0 con 0x7f201819d720 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.520+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201819e310 con 0x7f201819d720 2024-08-01T04:06:44.473 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.520+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20181a0df0 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.521+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f200c03d440 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.522+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2018106000 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.522+0000 7f2015600640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 0x7f200003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.522+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f200c06cab0 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.522+0000 7f2017e00640 1 -- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 msgr2=0x7f200003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.522+0000 7f2017e00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 0x7f200003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.532+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f200c045360 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.723+0000 7f2017e00640 1 -- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 msgr2=0x7f200003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.723+0000 7f2017e00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 0x7f200003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.763+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2018196a70 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.764+0000 7f2015600640 1 -- 172.21.15.3:0/4282260291 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f200c06c400 con 0x7f201819d720 2024-08-01T04:06:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 msgr2=0x7f200003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 0x7f200003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 msgr2=0x7f201819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 0x7f201819db40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f200c02a860 tx=0x7f200c030d40 comp rx=0 tx=0).stop 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 shutdown_connections 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7f2000039020 0x7f200003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 --2- 172.21.15.3:0/4282260291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f201819d720 0x7f201819db40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 >> 172.21.15.3:0/4282260291 conn(0x7f2018101a30 msgr2=0x7f2018191900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 shutdown_connections 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:43.767+0000 7f201de00640 1 -- 172.21.15.3:0/4282260291 wait complete. 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-08-01T04:06:44.475 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 9... 2024-08-01T04:06:44.716 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.553+0000 7f3b13840040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:06:44.717 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.629+0000 7f3b13840040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:06:44.717 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.714+0000 7f3b13840040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:06:45.010 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:44 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:44.897+0000 7f3b13840040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:06:45.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:45 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:45.447+0000 7f3b13840040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:06:46.080 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:45 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:45.823+0000 7f3b13840040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:06:46.080 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:45 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:45.904+0000 7f3b13840040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:06:46.430 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:46 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:46.089+0000 7f3b13840040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:06:46.776 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:46 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:46.467+0000 7f3b13840040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:06:46.776 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:46 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:46.543+0000 7f3b13840040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:06:46.776 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:46 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:46.773+0000 7f3b13840040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:06:47.047 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:46 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:46.965+0000 7f3b13840040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:06:47.047 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:47 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:47.044+0000 7f3b13840040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:06:47.298 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:47 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:47.145+0000 7f3b13840040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:06:47.298 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:47 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:47.218+0000 7f3b13840040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:06:48.298 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:47 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:47.861+0000 7f3b13840040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:06:48.298 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:48.050+0000 7f3b13840040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:06:48.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: Active manager daemon x restarted 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: Activating manager daemon x 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: osdmap e3: 0 total, 0 up, 0 in 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: mgrmap e10: x(active, starting, since 0.0116837s) 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: Manager daemon x is now available 2024-08-01T04:06:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:06:49.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:06:49.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:06:49.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:49 smithi003 ceph-mon[32262]: mgrmap e11: x(active, since 1.01014s) 2024-08-01T04:06:49.940 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-08-01T04:06:49.940 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-01T04:06:49.940 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-01T04:06:49.948 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-08-01T04:06:49.948 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.014+0000 7fd66e200640 1 Processor -- start 2024-08-01T04:06:49.948 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.015+0000 7fd66e200640 1 -- start start 2024-08-01T04:06:49.948 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.015+0000 7fd66e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:49.948 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.015+0000 7fd66e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd66807c7f0 con 0x7fd66807de00 2024-08-01T04:06:49.949 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.015+0000 7fd66ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:49.949 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.016+0000 7fd66ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49528/0 (socket says 172.21.15.3:49528) 2024-08-01T04:06:49.949 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.016+0000 7fd66ce00640 1 -- 172.21.15.3:0/3818146323 learned_addr learned my addr 172.21.15.3:0/3818146323 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:49.949 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.016+0000 7fd66ce00640 1 -- 172.21.15.3:0/3818146323 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66807c970 con 0x7fd66807de00 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.017+0000 7fd66ce00640 1 --2- 172.21.15.3:0/3818146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66807c2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009820 tx=0x7fd65c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f423a312cefbd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.017+0000 7fd667400640 1 -- 172.21.15.3:0/3818146323 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c037070 con 0x7fd66807de00 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.017+0000 7fd667400640 1 -- 172.21.15.3:0/3818146323 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c032920 con 0x7fd66807de00 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.017+0000 7fd667400640 1 -- 172.21.15.3:0/3818146323 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c030810 con 0x7fd66807de00 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.018+0000 7fd66e200640 1 -- 172.21.15.3:0/3818146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 msgr2=0x7fd66807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.018+0000 7fd66e200640 1 --2- 172.21.15.3:0/3818146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66807c2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009820 tx=0x7fd65c02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.018+0000 7fd66e200640 1 -- 172.21.15.3:0/3818146323 shutdown_connections 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.018+0000 7fd66e200640 1 --2- 172.21.15.3:0/3818146323 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66807c2b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:49.950 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.018+0000 7fd66e200640 1 -- 172.21.15.3:0/3818146323 >> 172.21.15.3:0/3818146323 conn(0x7fd668102ae0 msgr2=0x7fd668104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.019+0000 7fd66e200640 1 -- 172.21.15.3:0/3818146323 shutdown_connections 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.019+0000 7fd66e200640 1 -- 172.21.15.3:0/3818146323 wait complete. 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.019+0000 7fd66e200640 1 Processor -- start 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.020+0000 7fd66e200640 1 -- start start 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.020+0000 7fd66e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.020+0000 7fd66e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd66810d9c0 con 0x7fd66807de00 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.020+0000 7fd66ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.020+0000 7fd66ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49544/0 (socket says 172.21.15.3:49544) 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.020+0000 7fd66ce00640 1 -- 172.21.15.3:0/2719869359 learned_addr learned my addr 172.21.15.3:0/2719869359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.021+0000 7fd66ce00640 1 -- 172.21.15.3:0/2719869359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66819da50 con 0x7fd66807de00 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.021+0000 7fd66ce00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66819d510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd65c030380 tx=0x7fd65c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:49.951 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.021+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c037040 con 0x7fd66807de00 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.021+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c040070 con 0x7fd66807de00 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.021+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c03a7b0 con 0x7fd66807de00 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.021+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66819dce0 con 0x7fd66807de00 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.022+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a09d0 con 0x7fd66807de00 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.023+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd65c03bc00 con 0x7fd66807de00 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.023+0000 7fd665600640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.023+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 --> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd65003bbf0 con 0x7fd650039020 2024-08-01T04:06:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.024+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd65c06bd10 con 0x7fd66807de00 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.024+0000 7fd667e00640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 msgr2=0x7fd65003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.024+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.225+0000 7fd667e00640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 msgr2=0x7fd65003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.225+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.626+0000 7fd667e00640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 msgr2=0x7fd65003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:45.626+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:46.427+0000 7fd667e00640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 msgr2=0x7fd65003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:46.427+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-01T04:06:49.953 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:48.029+0000 7fd667e00640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 msgr2=0x7fd65003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2156560379 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:48.029+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:48.061+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fd65c03a950 con 0x7fd66807de00 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:48.061+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 msgr2=0x7fd65003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:48.061+0000 7fd665600640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.059+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fd65c06db40 con 0x7fd66807de00 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.060+0000 7fd665600640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd65003cae0 0x7fd65003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.060+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd65c007e50 con 0x7fd65003cae0 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.061+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd65003cae0 0x7fd65003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:49.954 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.063+0000 7fd667e00640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd65003cae0 0x7fd65003eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd658003980 tx=0x7fd6580092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.064+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd65c007e50 con 0x7fd65003cae0 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.072+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd66807c330 con 0x7fd65003cae0 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.072+0000 7fd665600640 1 -- 172.21.15.3:0/2719869359 <== mgr.14148 v2:172.21.15.3:6800/1617846146 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd66807c330 con 0x7fd65003cae0 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd65003cae0 msgr2=0x7fd65003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd65003cae0 0x7fd65003eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd658003980 tx=0x7fd6580092b0 comp rx=0 tx=0).stop 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 msgr2=0x7fd66819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66819d510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd65c030380 tx=0x7fd65c032670 comp rx=0 tx=0).stop 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 shutdown_connections 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd65003cae0 0x7fd65003eed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:49.955 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:6800/2156560379,v1:172.21.15.3:6801/2156560379] conn(0x7fd650039020 0x7fd65003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:49.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 --2- 172.21.15.3:0/2719869359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd66807de00 0x7fd66819d510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:49.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 >> 172.21.15.3:0/2719869359 conn(0x7fd668102ae0 msgr2=0x7fd6681036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:49.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 shutdown_connections 2024-08-01T04:06:49.956 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:49.073+0000 7fd66e200640 1 -- 172.21.15.3:0/2719869359 wait complete. 2024-08-01T04:06:49.956 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 9 is available 2024-08-01T04:06:49.956 INFO:teuthology.orchestra.run.smithi003.stdout:Generating a dashboard self-signed certificate... 2024-08-01T04:06:50.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-mon[32262]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-01T04:06:50.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-mon[32262]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-01T04:06:50.549 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:50.326+0000 7f3a6a600640 -1 log_channel(cephadm) log [ERR] : [01/Aug/2024:04:06:50] ENGINE Error in HTTPServer.serve 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: Traceback (most recent call last): 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._connections.run(self.expiration_interval) 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._run(expiration_interval) 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:06:50.550 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: s = self.context.wrap_socket( 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: return self.sslsocket_class._create( 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self.do_handshake() 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._sslobj.do_handshake() 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:06:50.551 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:06:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:06:51.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:50] ENGINE Bus STARTING 2024-08-01T04:06:51.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: mgrmap e12: x(active, since 2s) 2024-08-01T04:06:51.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:50] ENGINE Serving on http://172.21.15.3:8765 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:50] ENGINE Serving on https://172.21.15.3:7150 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:50] ENGINE Bus STARTED 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: [01/Aug/2024:04:06:50] ENGINE Error in HTTPServer.serve 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:06:51.300 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:06:51.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.766+0000 7fd110800640 1 Processor -- start 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.766+0000 7fd110800640 1 -- start start 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.767+0000 7fd110800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd108081940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.767+0000 7fd110800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd108081f10 con 0x7fd108081540 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.767+0000 7fd10de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd108081940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.767+0000 7fd10de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd108081940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49594/0 (socket says 172.21.15.3:49594) 2024-08-01T04:06:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.767+0000 7fd10de00640 1 -- 172.21.15.3:0/2388020299 learned_addr learned my addr 172.21.15.3:0/2388020299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.768+0000 7fd10de00640 1 -- 172.21.15.3:0/2388020299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10810ed60 con 0x7fd108081540 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.769+0000 7fd10de00640 1 --2- 172.21.15.3:0/2388020299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd108081940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009820 tx=0x7fd0fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453db652982438f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.769+0000 7fd10ca00640 1 -- 172.21.15.3:0/2388020299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc037070 con 0x7fd108081540 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.769+0000 7fd10ca00640 1 -- 172.21.15.3:0/2388020299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc032920 con 0x7fd108081540 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.769+0000 7fd10ca00640 1 -- 172.21.15.3:0/2388020299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc030810 con 0x7fd108081540 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 -- 172.21.15.3:0/2388020299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 msgr2=0x7fd108081940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 --2- 172.21.15.3:0/2388020299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd108081940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009820 tx=0x7fd0fc02a270 comp rx=0 tx=0).stop 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 -- 172.21.15.3:0/2388020299 shutdown_connections 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 --2- 172.21.15.3:0/2388020299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd108081940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 -- 172.21.15.3:0/2388020299 >> 172.21.15.3:0/2388020299 conn(0x7fd10807cd60 msgr2=0x7fd10807f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 -- 172.21.15.3:0/2388020299 shutdown_connections 2024-08-01T04:06:51.727 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.770+0000 7fd110800640 1 -- 172.21.15.3:0/2388020299 wait complete. 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.771+0000 7fd110800640 1 Processor -- start 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.772+0000 7fd110800640 1 -- start start 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.772+0000 7fd110800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd1081a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.772+0000 7fd110800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd108112110 con 0x7fd108081540 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.772+0000 7fd10de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd1081a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.772+0000 7fd10de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd1081a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49600/0 (socket says 172.21.15.3:49600) 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.772+0000 7fd10de00640 1 -- 172.21.15.3:0/3985085563 learned_addr learned my addr 172.21.15.3:0/3985085563 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:51.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd10de00640 1 -- 172.21.15.3:0/3985085563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1081a20f0 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd10de00640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd1081a1bb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097f0 tx=0x7fd0fc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc037040 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc030c40 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0fc03a5b0 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1081a2380 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.773+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081a5070 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.775+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd108112290 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.775+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fd0fc03a750 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.775+0000 7fd102a00640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd0f0039020 0x7fd0f003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.776+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd0fc06c4f0 con 0x7fd108081540 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.776+0000 7fd10d400640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd0f0039020 0x7fd0f003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:51.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.777+0000 7fd10d400640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd0f0039020 0x7fd0f003b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80098c0 tx=0x7fd0f8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.781+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd0fc042e10 con 0x7fd108081540 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:50.985+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd10807e030 con 0x7fd0f0039020 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.096+0000 7fd102a00640 1 -- 172.21.15.3:0/3985085563 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd10807e030 con 0x7fd0f0039020 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd0f0039020 msgr2=0x7fd0f003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd0f0039020 0x7fd0f003b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80098c0 tx=0x7fd0f8006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 msgr2=0x7fd1081a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd1081a1bb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0097f0 tx=0x7fd0fc0327b0 comp rx=0 tx=0).stop 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 shutdown_connections 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd0f0039020 0x7fd0f003b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 --2- 172.21.15.3:0/3985085563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd108081540 0x7fd1081a1bb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 >> 172.21.15.3:0/3985085563 conn(0x7fd10807cd60 msgr2=0x7fd10807d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 shutdown_connections 2024-08-01T04:06:51.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:51.099+0000 7fd110800640 1 -- 172.21.15.3:0/3985085563 wait complete. 2024-08-01T04:06:51.731 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2024-08-01T04:06:52.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:52 smithi003 ceph-mon[32262]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:53.427 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$a1TDFWwLAvyYiYcZB5cy0OLVWqRxQh3ul6uEw.LYamx6I0E.8gePC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1722485212, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-01T04:06:53.427 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.307+0000 7f4ba5c00640 1 Processor -- start 2024-08-01T04:06:53.427 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.307+0000 7f4ba5c00640 1 -- start start 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.307+0000 7f4ba5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba0107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.307+0000 7f4ba5c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0107f00 con 0x7f4ba0107530 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.308+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba0107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.308+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba0107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54200/0 (socket says 172.21.15.3:54200) 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.308+0000 7f4b9ea00640 1 -- 172.21.15.3:0/560787207 learned_addr learned my addr 172.21.15.3:0/560787207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.308+0000 7f4b9ea00640 1 -- 172.21.15.3:0/560787207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba0108730 con 0x7f4ba0107530 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.310+0000 7f4b9ea00640 1 --2- 172.21.15.3:0/560787207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba0107930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009e80 tx=0x7f4b9002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1be7a2eb997d00a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.310+0000 7f4b9d600640 1 -- 172.21.15.3:0/560787207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b90036070 con 0x7f4ba0107530 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.310+0000 7f4b9d600640 1 -- 172.21.15.3:0/560787207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b900318e0 con 0x7f4ba0107530 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.310+0000 7f4b9d600640 1 -- 172.21.15.3:0/560787207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b90007a00 con 0x7f4ba0107530 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 -- 172.21.15.3:0/560787207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 msgr2=0x7f4ba0107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:53.428 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 --2- 172.21.15.3:0/560787207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba0107930 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009e80 tx=0x7f4b9002a7a0 comp rx=0 tx=0).stop 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 -- 172.21.15.3:0/560787207 shutdown_connections 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 --2- 172.21.15.3:0/560787207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba0107930 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 -- 172.21.15.3:0/560787207 >> 172.21.15.3:0/560787207 conn(0x7f4ba0102d00 msgr2=0x7f4ba0105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 -- 172.21.15.3:0/560787207 shutdown_connections 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.311+0000 7f4ba5c00640 1 -- 172.21.15.3:0/560787207 wait complete. 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.313+0000 7f4ba5c00640 1 Processor -- start 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.313+0000 7f4ba5c00640 1 -- start start 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.314+0000 7f4ba5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba007db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.314+0000 7f4ba5c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba010bae0 con 0x7f4ba0107530 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.314+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba007db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.314+0000 7f4b9ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba007db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54204/0 (socket says 172.21.15.3:54204) 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.314+0000 7f4b9ea00640 1 -- 172.21.15.3:0/3113982690 learned_addr learned my addr 172.21.15.3:0/3113982690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:53.429 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.314+0000 7f4b9ea00640 1 -- 172.21.15.3:0/3113982690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba007e0a0 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.315+0000 7f4b9ea00640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba007db60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4b9002acc0 tx=0x7f4b90031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.315+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b90036050 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.315+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b90037040 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.315+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba007c290 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.315+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba007c7d0 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.315+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b9003b890 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.316+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4b90044020 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.317+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b800052c0 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.317+0000 7f4b97400640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b84039020 0x7f4b8403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.317+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b90042020 con 0x7f4ba0107530 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.317+0000 7f4b9e000640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b84039020 0x7f4b8403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:53.430 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.318+0000 7f4b9e000640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b84039020 0x7f4b8403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098c0 tx=0x7f4b8c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.322+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4b9004b020 con 0x7f4ba0107530 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.535+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4b80002420 con 0x7f4b84039020 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.795+0000 7f4b97400640 1 -- 172.21.15.3:0/3113982690 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4b80002420 con 0x7f4b84039020 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b84039020 msgr2=0x7f4b8403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b84039020 0x7f4b8403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098c0 tx=0x7f4b8c006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 msgr2=0x7f4ba007db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba007db60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4b9002acc0 tx=0x7f4b90031cc0 comp rx=0 tx=0).stop 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 shutdown_connections 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b84039020 0x7f4b8403b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 --2- 172.21.15.3:0/3113982690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ba0107530 0x7f4ba007db60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.798+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 >> 172.21.15.3:0/3113982690 conn(0x7f4ba0102d00 msgr2=0x7f4ba0103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:53.431 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.799+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 shutdown_connections 2024-08-01T04:06:53.432 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:52.799+0000 7f4ba5c00640 1 -- 172.21.15.3:0/3113982690 wait complete. 2024-08-01T04:06:53.432 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2024-08-01T04:06:53.699 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:06:53.699 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:53 smithi003 ceph-mon[32262]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:06:53.699 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:06:55.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:55 smithi003 ceph-mon[32262]: mgrmap e13: x(active, since 6s) 2024-08-01T04:06:55.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:55 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1374856834' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.241+0000 7ff603800640 1 Processor -- start 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.241+0000 7ff603800640 1 -- start start 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.242+0000 7ff603800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.242+0000 7ff603800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc07c8a0 con 0x7ff5fc07de70 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.242+0000 7ff600e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.242+0000 7ff600e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54212/0 (socket says 172.21.15.3:54212) 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.242+0000 7ff600e00640 1 -- 172.21.15.3:0/3913612079 learned_addr learned my addr 172.21.15.3:0/3913612079 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.243+0000 7ff600e00640 1 -- 172.21.15.3:0/3913612079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc07ca20 con 0x7ff5fc07de70 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.244+0000 7ff600e00640 1 --2- 172.21.15.3:0/3913612079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc07c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e80 tx=0x7ff5ec02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177cb18318e1b29c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:55.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.244+0000 7ff5fb400640 1 -- 172.21.15.3:0/3913612079 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec036070 con 0x7ff5fc07de70 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.244+0000 7ff5fb400640 1 -- 172.21.15.3:0/3913612079 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec0318e0 con 0x7ff5fc07de70 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.244+0000 7ff5fb400640 1 -- 172.21.15.3:0/3913612079 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec007a00 con 0x7ff5fc07de70 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 -- 172.21.15.3:0/3913612079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 msgr2=0x7ff5fc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 --2- 172.21.15.3:0/3913612079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc07c2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e80 tx=0x7ff5ec02a7a0 comp rx=0 tx=0).stop 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 -- 172.21.15.3:0/3913612079 shutdown_connections 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 --2- 172.21.15.3:0/3913612079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc07c2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 -- 172.21.15.3:0/3913612079 >> 172.21.15.3:0/3913612079 conn(0x7ff5fc102ce0 msgr2=0x7ff5fc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 -- 172.21.15.3:0/3913612079 shutdown_connections 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.245+0000 7ff603800640 1 -- 172.21.15.3:0/3913612079 wait complete. 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.246+0000 7ff603800640 1 Processor -- start 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.247+0000 7ff603800640 1 -- start start 2024-08-01T04:06:55.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.247+0000 7ff603800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.247+0000 7ff603800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc10dba0 con 0x7ff5fc07de70 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.247+0000 7ff600e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.247+0000 7ff600e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc19d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54216/0 (socket says 172.21.15.3:54216) 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.247+0000 7ff600e00640 1 -- 172.21.15.3:0/1374856834 learned_addr learned my addr 172.21.15.3:0/1374856834 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.248+0000 7ff600e00640 1 -- 172.21.15.3:0/1374856834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19dc50 con 0x7ff5fc07de70 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.248+0000 7ff600e00640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc19d710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02acc0 tx=0x7ff5ec031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.248+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec036050 con 0x7ff5fc07de70 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.248+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec037040 con 0x7ff5fc07de70 2024-08-01T04:06:55.651 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.248+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19dee0 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.248+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec03d8e0 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.249+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a0bd0 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.250+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff5ec044020 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.250+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc07c350 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.250+0000 7ff5f9600640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5e0039020 0x7ff5e003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.250+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff5ec042020 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.250+0000 7ff5fbe00640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5e0039020 0x7ff5e003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.251+0000 7ff5fbe00640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5e0039020 0x7ff5e003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098c0 tx=0x7ff5e4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.256+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff5ec04b020 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.378+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff5ec04bc90 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.452+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff5fc1a11c0 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.453+0000 7ff5f9600640 1 -- 172.21.15.3:0/1374856834 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff5ec0432c0 con 0x7ff5fc07de70 2024-08-01T04:06:55.652 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5e0039020 msgr2=0x7ff5e003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5e0039020 0x7ff5e003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098c0 tx=0x7ff5e4006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 msgr2=0x7ff5fc19d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc19d710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec02acc0 tx=0x7ff5ec031cc0 comp rx=0 tx=0).stop 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 shutdown_connections 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5e0039020 0x7ff5e003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 --2- 172.21.15.3:0/1374856834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5fc07de70 0x7ff5fc19d710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.456+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 >> 172.21.15.3:0/1374856834 conn(0x7ff5fc102ce0 msgr2=0x7ff5fc103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.457+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 shutdown_connections 2024-08-01T04:06:55.653 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:54.457+0000 7ff603800640 1 -- 172.21.15.3:0/1374856834 wait complete. 2024-08-01T04:06:55.682 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-01T04:06:55.683 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-08-01T04:06:55.708 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-01T04:06:55.708 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-08-01T04:06:55.708 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-08-01T04:06:55.709 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-01T04:06:55.716 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2024-08-01T04:06:55.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:55.716 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2024-08-01T04:06:55.716 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2024-08-01T04:06:55.716 INFO:teuthology.orchestra.run.smithi003.stdout: Password: a8f4bqpdpb 2024-08-01T04:06:55.716 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:55.717 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config directory 2024-08-01T04:06:57.396 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.426+0000 7f2807200640 1 Processor -- start 2024-08-01T04:06:57.396 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.426+0000 7f2807200640 1 -- start start 2024-08-01T04:06:57.397 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.426+0000 7f2807200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f2808109c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:57.397 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.426+0000 7f2807200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f280810a230 con 0x7f2808109860 2024-08-01T04:06:57.397 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.427+0000 7f2805e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f2808109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:57.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.427+0000 7f2805e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f2808109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54230/0 (socket says 172.21.15.3:54230) 2024-08-01T04:06:57.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.427+0000 7f2805e00640 1 -- 172.21.15.3:0/382321484 learned_addr learned my addr 172.21.15.3:0/382321484 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:57.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.427+0000 7f2805e00640 1 -- 172.21.15.3:0/382321484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280810a740 con 0x7f2808109860 2024-08-01T04:06:57.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.428+0000 7f2805e00640 1 --2- 172.21.15.3:0/382321484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f2808109c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009820 tx=0x7f27f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb35e0aaaf8d3fa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:57.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.429+0000 7f2804a00640 1 -- 172.21.15.3:0/382321484 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8037070 con 0x7f2808109860 2024-08-01T04:06:57.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.429+0000 7f2804a00640 1 -- 172.21.15.3:0/382321484 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f8032920 con 0x7f2808109860 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.429+0000 7f2804a00640 1 -- 172.21.15.3:0/382321484 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8030810 con 0x7f2808109860 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 -- 172.21.15.3:0/382321484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 msgr2=0x7f2808109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 --2- 172.21.15.3:0/382321484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f2808109c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009820 tx=0x7f27f802a270 comp rx=0 tx=0).stop 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 -- 172.21.15.3:0/382321484 shutdown_connections 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 --2- 172.21.15.3:0/382321484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f2808109c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 -- 172.21.15.3:0/382321484 >> 172.21.15.3:0/382321484 conn(0x7f280807cb20 msgr2=0x7f280807cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 -- 172.21.15.3:0/382321484 shutdown_connections 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.430+0000 7f2807200640 1 -- 172.21.15.3:0/382321484 wait complete. 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.431+0000 7f2807200640 1 Processor -- start 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.432+0000 7f2807200640 1 -- start start 2024-08-01T04:06:57.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.432+0000 7f2807200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f280819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.432+0000 7f2807200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f280810daf0 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.432+0000 7f2805e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f280819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.432+0000 7f2805e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f280819d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54244/0 (socket says 172.21.15.3:54244) 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.432+0000 7f2805e00640 1 -- 172.21.15.3:0/1167909023 learned_addr learned my addr 172.21.15.3:0/1167909023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.433+0000 7f2805e00640 1 -- 172.21.15.3:0/1167909023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280819da80 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.433+0000 7f2805e00640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f280819d540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097f0 tx=0x7f27f80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.433+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8037040 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.433+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f8030c40 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.433+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f803a5d0 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.433+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280819dd10 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.434+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081a0a00 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.435+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2808109ce0 con 0x7f2808109860 2024-08-01T04:06:57.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.435+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f27f803b3f0 con 0x7f2808109860 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.435+0000 7f27fea00640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27ec0390c0 0x7f27ec03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.436+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27f806c6d0 con 0x7f2808109860 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.436+0000 7f2805400640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27ec0390c0 0x7f27ec03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.436+0000 7f2805400640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27ec0390c0 0x7f27ec03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098c0 tx=0x7f27f4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.441+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f27f8041510 con 0x7f2808109860 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.730+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f28081a0ff0 con 0x7f2808109860 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.734+0000 7f27fea00640 1 -- 172.21.15.3:0/1167909023 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f27f80412b0 con 0x7f2808109860 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-01T04:06:57.405 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27ec0390c0 msgr2=0x7f27ec03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27ec0390c0 0x7f27ec03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098c0 tx=0x7f27f4006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 msgr2=0x7f280819d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f280819d540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097f0 tx=0x7f27f80327b0 comp rx=0 tx=0).stop 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 shutdown_connections 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27ec0390c0 0x7f27ec03b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 --2- 172.21.15.3:0/1167909023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2808109860 0x7f280819d540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 >> 172.21.15.3:0/1167909023 conn(0x7f280807cb20 msgr2=0x7f2808107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.737+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 shutdown_connections 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-08-01T04:06:56.738+0000 7f2807200640 1 -- 172.21.15.3:0/1167909023 wait complete. 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.406 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:06:57.407 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2024-08-01T04:06:57.517 INFO:tasks.cephadm:Fetching config... 2024-08-01T04:06:57.517 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:06:57.518 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-01T04:06:57.566 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-01T04:06:57.566 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:06:57.566 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-01T04:06:57.647 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-01T04:06:57.647 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:06:57.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-01T04:06:57.778 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-01T04:06:57.778 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:06:57.778 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-01T04:06:57.836 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-01T04:06:57.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-01T04:06:57.938 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:06:57.969 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:06:57 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1167909023' entity='client.admin' 2024-08-01T04:06:57.986 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-01T04:06:58.029 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:06:58.042 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-01T04:06:58.084 INFO:teuthology.orchestra.run.smithi163.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw9e0QONH54O6PpnS7XV31b5gLPxTEQPd4IIKvlNF3/SB0/YeZQPIOUHQUv6T7q3fPjPFx91JNLPM2HepfKgti1utlWxzaJI8E0pEnga2ouun+eiG6OdJmQRjNc44UkILJQZ4lQWHJCHM3qICoG6TfyvOQp0VRjWr0dtdBF9rU+EEYaqD/R2ZuwHlddZu4FrqZ9zsB4ZK4a/nW7EBdLa6duA9nM20yThOIC/G8xDhepOOfEzbYiWV4WobcqISd7CVts6HxAFnVgYn0a3k/HcOr8qvJ7+6bzlRvA/hiE8GTCNVsic8sDCrxZA07Yp/+6Xi6BKwK2C2yCxb0jaJWVMIAIeapN7XbSP1n2rDzeusXcUeqQGe+ebTgx4V8o22BC3tozCjFpKvEIXyhyDVtXZhsvmowTdc/rAXwutfjLE7H4Hwb0Y0MgUSRD3y1xHd4Yhc4Fo3gmAuh5D84f22uzxN6jG6M/vCDDW7/lwR1+tS/axn3nL2do7L4L/DXtMbJMts= ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:06:58.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-01T04:06:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:06:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 -- 172.21.15.3:0/882276266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 msgr2=0x7f60c0103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 --2- 172.21.15.3:0/882276266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0103770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009880 tx=0x7f60b402a5d0 comp rx=0 tx=0).stop 2024-08-01T04:06:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 -- 172.21.15.3:0/882276266 shutdown_connections 2024-08-01T04:06:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 --2- 172.21.15.3:0/882276266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0103770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 -- 172.21.15.3:0/882276266 >> 172.21.15.3:0/882276266 conn(0x7f60c00feb00 msgr2=0x7f60c0100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:59.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 -- 172.21.15.3:0/882276266 shutdown_connections 2024-08-01T04:06:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.137+0000 7f60c6800640 1 -- 172.21.15.3:0/882276266 wait complete. 2024-08-01T04:06:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.138+0000 7f60c6800640 1 Processor -- start 2024-08-01T04:06:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.138+0000 7f60c6800640 1 -- start start 2024-08-01T04:06:59.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.139+0000 7f60c6800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.139+0000 7f60c6800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f60c0109480 con 0x7f60c0103370 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.139+0000 7f60c5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.139+0000 7f60c5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54256/0 (socket says 172.21.15.3:54256) 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.139+0000 7f60c5400640 1 -- 172.21.15.3:0/3082740574 learned_addr learned my addr 172.21.15.3:0/3082740574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.140+0000 7f60c5400640 1 -- 172.21.15.3:0/3082740574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c0199030 con 0x7f60c0103370 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.140+0000 7f60c5400640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0198af0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60b4004f60 tx=0x7f60b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.140+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60b403f070 con 0x7f60c0103370 2024-08-01T04:06:59.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.140+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60b4004cc0 con 0x7f60c0103370 2024-08-01T04:06:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.140+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60b4035030 con 0x7f60c0103370 2024-08-01T04:06:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.140+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c01992c0 con 0x7f60c0103370 2024-08-01T04:06:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.141+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c019bfb0 con 0x7f60c0103370 2024-08-01T04:06:59.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.142+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f60b4042070 con 0x7f60c0103370 2024-08-01T04:06:59.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.142+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c0109c00 con 0x7f60c0103370 2024-08-01T04:06:59.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.142+0000 7f60be000640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f60a80390c0 0x7f60a803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:06:59.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.143+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f60b406cdd0 con 0x7f60c0103370 2024-08-01T04:06:59.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.143+0000 7f60c4a00640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f60a80390c0 0x7f60a803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:06:59.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.143+0000 7f60c4a00640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f60a80390c0 0x7f60a803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098c0 tx=0x7f60ac006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:06:59.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.151+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f60b4040260 con 0x7f60c0103370 2024-08-01T04:06:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.348+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f60c002d650 con 0x7f60c0103370 2024-08-01T04:06:59.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.354+0000 7f60be000640 1 -- 172.21.15.3:0/3082740574 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f60b4003dd0 con 0x7f60c0103370 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f60a80390c0 msgr2=0x7f60a803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f60a80390c0 0x7f60a803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098c0 tx=0x7f60ac006db0 comp rx=0 tx=0).stop 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 msgr2=0x7f60c0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0198af0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60b4004f60 tx=0x7f60b40048f0 comp rx=0 tx=0).stop 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 shutdown_connections 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f60a80390c0 0x7f60a803b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 --2- 172.21.15.3:0/3082740574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60c0103370 0x7f60c0198af0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:06:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.357+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 >> 172.21.15.3:0/3082740574 conn(0x7f60c00feb00 msgr2=0x7f60c00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:06:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.358+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 shutdown_connections 2024-08-01T04:06:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:06:59.358+0000 7f60c6800640 1 -- 172.21.15.3:0/3082740574 wait complete. 2024-08-01T04:07:00.683 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:00 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3082740574' entity='client.admin' 2024-08-01T04:07:00.739 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-01T04:07:00.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-01T04:07:01.381 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 -- 172.21.15.3:0/1056628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 msgr2=0x7f6458101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 --2- 172.21.15.3:0/1056628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458101570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6448009880 tx=0x7f644802a5d0 comp rx=0 tx=0).stop 2024-08-01T04:07:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 -- 172.21.15.3:0/1056628371 shutdown_connections 2024-08-01T04:07:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 --2- 172.21.15.3:0/1056628371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458101570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 -- 172.21.15.3:0/1056628371 >> 172.21.15.3:0/1056628371 conn(0x7f64580fc940 msgr2=0x7f64580fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 -- 172.21.15.3:0/1056628371 shutdown_connections 2024-08-01T04:07:02.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.006+0000 7f6457200640 1 -- 172.21.15.3:0/1056628371 wait complete. 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.007+0000 7f6457200640 1 Processor -- start 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.008+0000 7f6457200640 1 -- start start 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.008+0000 7f6457200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.008+0000 7f6457200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6458074850 con 0x7f6458101170 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.008+0000 7f6455e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.008+0000 7f6455e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55488/0 (socket says 172.21.15.3:55488) 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.008+0000 7f6455e00640 1 -- 172.21.15.3:0/1952965021 learned_addr learned my addr 172.21.15.3:0/1952965021 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.009+0000 7f6455e00640 1 -- 172.21.15.3:0/1952965021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6458199040 con 0x7f6458101170 2024-08-01T04:07:02.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.009+0000 7f6455e00640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458198b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6448004f60 tx=0x7f64480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:02.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.009+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f644803f070 con 0x7f6458101170 2024-08-01T04:07:02.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.009+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6448004cc0 con 0x7f6458101170 2024-08-01T04:07:02.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.009+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64581992d0 con 0x7f6458101170 2024-08-01T04:07:02.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.010+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645819bfc0 con 0x7f6458101170 2024-08-01T04:07:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.010+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6448035030 con 0x7f6458101170 2024-08-01T04:07:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.011+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6458074fd0 con 0x7f6458101170 2024-08-01T04:07:02.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.011+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f6448042070 con 0x7f6458101170 2024-08-01T04:07:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.012+0000 7f644ea00640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f643c039060 0x7f643c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.012+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f644806cec0 con 0x7f6458101170 2024-08-01T04:07:02.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.012+0000 7f6455400640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f643c039060 0x7f643c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:02.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.013+0000 7f6455400640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f643c039060 0x7f643c03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64440098c0 tx=0x7f6444006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:02.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.017+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f64480433b0 con 0x7f6458101170 2024-08-01T04:07:02.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.217+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6458062360 con 0x7f643c039060 2024-08-01T04:07:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.222+0000 7f644ea00640 1 -- 172.21.15.3:0/1952965021 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6458062360 con 0x7f643c039060 2024-08-01T04:07:02.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f643c039060 msgr2=0x7f643c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f643c039060 0x7f643c03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64440098c0 tx=0x7f6444006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 msgr2=0x7f6458198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458198b00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6448004f60 tx=0x7f64480048f0 comp rx=0 tx=0).stop 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 shutdown_connections 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f643c039060 0x7f643c03b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 --2- 172.21.15.3:0/1952965021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6458101170 0x7f6458198b00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.224+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 >> 172.21.15.3:0/1952965021 conn(0x7f64580fc940 msgr2=0x7f6458104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.225+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 shutdown_connections 2024-08-01T04:07:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:02.225+0000 7f6457200640 1 -- 172.21.15.3:0/1952965021 wait complete. 2024-08-01T04:07:03.185 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi160 2024-08-01T04:07:03.185 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:07:03.185 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.conf 2024-08-01T04:07:03.204 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:07:03.204 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:03.261 INFO:tasks.cephadm:Adding host smithi160 to orchestrator... 2024-08-01T04:07:03.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch host add smithi160 2024-08-01T04:07:03.343 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:03 smithi003 ceph-mon[32262]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:03.343 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:03 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:03.738 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 -- 172.21.15.3:0/1034058084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 msgr2=0x7fa130103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 --2- 172.21.15.3:0/1034058084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130103920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa120009880 tx=0x7fa12002a5d0 comp rx=0 tx=0).stop 2024-08-01T04:07:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 -- 172.21.15.3:0/1034058084 shutdown_connections 2024-08-01T04:07:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 --2- 172.21.15.3:0/1034058084 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130103920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 -- 172.21.15.3:0/1034058084 >> 172.21.15.3:0/1034058084 conn(0x7fa1300fecf0 msgr2=0x7fa130101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 -- 172.21.15.3:0/1034058084 shutdown_connections 2024-08-01T04:07:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.351+0000 7fa136600640 1 -- 172.21.15.3:0/1034058084 wait complete. 2024-08-01T04:07:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.353+0000 7fa136600640 1 Processor -- start 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.353+0000 7fa136600640 1 -- start start 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.353+0000 7fa136600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.353+0000 7fa136600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa130109630 con 0x7fa130103520 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.354+0000 7fa12f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.354+0000 7fa12f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55512/0 (socket says 172.21.15.3:55512) 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.354+0000 7fa12f400640 1 -- 172.21.15.3:0/437299803 learned_addr learned my addr 172.21.15.3:0/437299803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:04.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.354+0000 7fa12f400640 1 -- 172.21.15.3:0/437299803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1301991a0 con 0x7fa130103520 2024-08-01T04:07:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.354+0000 7fa12f400640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130198c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa120004f60 tx=0x7fa1200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.355+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa12003f070 con 0x7fa130103520 2024-08-01T04:07:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.355+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa130199430 con 0x7fa130103520 2024-08-01T04:07:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.355+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa120004cc0 con 0x7fa130103520 2024-08-01T04:07:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.355+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa120035030 con 0x7fa130103520 2024-08-01T04:07:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.355+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13019c120 con 0x7fa130103520 2024-08-01T04:07:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.356+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa120042070 con 0x7fa130103520 2024-08-01T04:07:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.356+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa130109db0 con 0x7fa130103520 2024-08-01T04:07:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.357+0000 7fa127e00640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa1140390c0 0x7fa11403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.357+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa12006cdf0 con 0x7fa130103520 2024-08-01T04:07:04.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.358+0000 7fa12ea00640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa1140390c0 0x7fa11403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:04.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.358+0000 7fa12ea00640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa1140390c0 0x7fa11403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098c0 tx=0x7fa11c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.364+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa1200433b0 con 0x7fa130103520 2024-08-01T04:07:04.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:04.568+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}) -- 0x7fa13002d650 con 0x7fa1140390c0 2024-08-01T04:07:04.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:04.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:04.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:07:04.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:04.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:04.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:04 smithi003 ceph-mon[32262]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:07:05.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:05 smithi003 ceph-mon[32262]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:05.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:05 smithi003 ceph-mon[32262]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:05.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:05 smithi003 ceph-mon[32262]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: Deploying cephadm binary to smithi160 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:07.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:07.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.861+0000 7fa127e00640 1 -- 172.21.15.3:0/437299803 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa13002d650 con 0x7fa1140390c0 2024-08-01T04:07:08.864 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi160' with addr '172.21.15.160' 2024-08-01T04:07:08.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa1140390c0 msgr2=0x7fa11403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:08.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa1140390c0 0x7fa11403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0098c0 tx=0x7fa11c006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 msgr2=0x7fa130198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130198c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa120004f60 tx=0x7fa1200048f0 comp rx=0 tx=0).stop 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 shutdown_connections 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa1140390c0 0x7fa11403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 --2- 172.21.15.3:0/437299803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa130103520 0x7fa130198c60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.866+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 >> 172.21.15.3:0/437299803 conn(0x7fa1300fecf0 msgr2=0x7fa1300ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.867+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 shutdown_connections 2024-08-01T04:07:08.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:08.867+0000 7fa136600640 1 -- 172.21.15.3:0/437299803 wait complete. 2024-08-01T04:07:09.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch host ls --format=json 2024-08-01T04:07:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:10.195 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:09 smithi003 ceph-mon[32262]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:10.195 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:10.196 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:09 smithi003 ceph-mon[32262]: Added host smithi160 2024-08-01T04:07:10.196 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:10.196 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:10.196 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 -- 172.21.15.3:0/3989255775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 msgr2=0x7fa9300746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 --2- 172.21.15.3:0/3989255775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa9300746a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa920009880 tx=0x7fa92002a670 comp rx=0 tx=0).stop 2024-08-01T04:07:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 -- 172.21.15.3:0/3989255775 shutdown_connections 2024-08-01T04:07:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 --2- 172.21.15.3:0/3989255775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa9300746a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 -- 172.21.15.3:0/3989255775 >> 172.21.15.3:0/3989255775 conn(0x7fa930077fa0 msgr2=0x7fa9300ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 -- 172.21.15.3:0/3989255775 shutdown_connections 2024-08-01T04:07:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.683+0000 7fa935000640 1 -- 172.21.15.3:0/3989255775 wait complete. 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.684+0000 7fa935000640 1 Processor -- start 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.684+0000 7fa935000640 1 -- start start 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.684+0000 7fa935000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa93019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.684+0000 7fa935000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa930111ef0 con 0x7fa9300742c0 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.685+0000 7fa92f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa93019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.685+0000 7fa92f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa93019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55534/0 (socket says 172.21.15.3:55534) 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.685+0000 7fa92f400640 1 -- 172.21.15.3:0/209566116 learned_addr learned my addr 172.21.15.3:0/209566116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.685+0000 7fa92f400640 1 -- 172.21.15.3:0/209566116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93019d930 con 0x7fa9300742c0 2024-08-01T04:07:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.685+0000 7fa92f400640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa93019d3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa920009230 tx=0x7fa920003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.686+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa92003f070 con 0x7fa9300742c0 2024-08-01T04:07:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.686+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92003bc80 con 0x7fa9300742c0 2024-08-01T04:07:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.686+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa920035030 con 0x7fa9300742c0 2024-08-01T04:07:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.686+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93019dbc0 con 0x7fa9300742c0 2024-08-01T04:07:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.686+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93019e020 con 0x7fa9300742c0 2024-08-01T04:07:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.687+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa920042070 con 0x7fa9300742c0 2024-08-01T04:07:10.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.687+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa930075a00 con 0x7fa9300742c0 2024-08-01T04:07:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.688+0000 7fa92cc00640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9140390c0 0x7fa91403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.688+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa92006cd80 con 0x7fa9300742c0 2024-08-01T04:07:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.688+0000 7fa927e00640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9140390c0 0x7fa91403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.689+0000 7fa927e00640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9140390c0 0x7fa91403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098c0 tx=0x7fa918006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.693+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa9200433b0 con 0x7fa9300742c0 2024-08-01T04:07:10.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.890+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa93019e750 con 0x7fa9140390c0 2024-08-01T04:07:10.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.893+0000 7fa92cc00640 1 -- 172.21.15.3:0/209566116 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fa93019e750 con 0x7fa9140390c0 2024-08-01T04:07:10.895 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:07:10.895 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}] 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9140390c0 msgr2=0x7fa91403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9140390c0 0x7fa91403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098c0 tx=0x7fa918006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 msgr2=0x7fa93019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa93019d3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa920009230 tx=0x7fa920003d40 comp rx=0 tx=0).stop 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 shutdown_connections 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9140390c0 0x7fa91403b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 --2- 172.21.15.3:0/209566116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9300742c0 0x7fa93019d3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.896+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 >> 172.21.15.3:0/209566116 conn(0x7fa930077fa0 msgr2=0x7fa930102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.897+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 shutdown_connections 2024-08-01T04:07:10.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:10.897+0000 7fa935000640 1 -- 172.21.15.3:0/209566116 wait complete. 2024-08-01T04:07:11.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:11 smithi003 ceph-mon[32262]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:11.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:11 smithi003 ceph-mon[32262]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:07:11.653 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi163 2024-08-01T04:07:11.653 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:07:11.653 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.conf 2024-08-01T04:07:11.671 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:07:11.671 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:11.727 INFO:tasks.cephadm:Adding host smithi163 to orchestrator... 2024-08-01T04:07:11.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch host add smithi163 2024-08-01T04:07:12.208 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 -- 172.21.15.3:0/2978916417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 msgr2=0x7f6b6c072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 --2- 172.21.15.3:0/2978916417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c072dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009880 tx=0x7f6b5402a5d0 comp rx=0 tx=0).stop 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 -- 172.21.15.3:0/2978916417 shutdown_connections 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 --2- 172.21.15.3:0/2978916417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c072dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 -- 172.21.15.3:0/2978916417 >> 172.21.15.3:0/2978916417 conn(0x7f6b6c06ca30 msgr2=0x7f6b6c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 -- 172.21.15.3:0/2978916417 shutdown_connections 2024-08-01T04:07:12.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.805+0000 7f6b70c00640 1 -- 172.21.15.3:0/2978916417 wait complete. 2024-08-01T04:07:12.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.806+0000 7f6b70c00640 1 Processor -- start 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.807+0000 7f6b70c00640 1 -- start start 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.807+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c1a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.807+0000 7f6b70c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c116450 con 0x7f6b6c10bb00 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.807+0000 7f6b69e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c1a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.807+0000 7f6b69e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c1a7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34186/0 (socket says 172.21.15.3:34186) 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.807+0000 7f6b69e00640 1 -- 172.21.15.3:0/1701973125 learned_addr learned my addr 172.21.15.3:0/1701973125 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:12.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.808+0000 7f6b69e00640 1 -- 172.21.15.3:0/1701973125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c1a7ba0 con 0x7f6b6c10bb00 2024-08-01T04:07:12.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.808+0000 7f6b69e00640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c1a7660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009230 tx=0x7f6b54003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:12.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.808+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b5403f070 con 0x7f6b6c10bb00 2024-08-01T04:07:12.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.808+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b5403bd40 con 0x7f6b6c10bb00 2024-08-01T04:07:12.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.808+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b54035030 con 0x7f6b6c10bb00 2024-08-01T04:07:12.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.809+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c1a7e30 con 0x7f6b6c10bb00 2024-08-01T04:07:12.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.809+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c1a1730 con 0x7f6b6c10bb00 2024-08-01T04:07:12.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.810+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c109130 con 0x7f6b6c10bb00 2024-08-01T04:07:12.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.810+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f6b54042070 con 0x7f6b6c10bb00 2024-08-01T04:07:12.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.810+0000 7f6b62a00640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6b500390c0 0x7f6b5003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.811+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6b5406ccd0 con 0x7f6b6c10bb00 2024-08-01T04:07:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.811+0000 7f6b69400640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6b500390c0 0x7f6b5003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:12.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.811+0000 7f6b69400640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6b500390c0 0x7f6b5003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0098c0 tx=0x7f6b5c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:12.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:12.816+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6b540412c0 con 0x7f6b6c10bb00 2024-08-01T04:07:13.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:13.020+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}) -- 0x7f6b6c1a2c60 con 0x7f6b500390c0 2024-08-01T04:07:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:13.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:07:13.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:13.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:14.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:14 smithi003 ceph-mon[32262]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:14.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:14 smithi003 ceph-mon[32262]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:07:14.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:14 smithi003 ceph-mon[32262]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:14.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:14 smithi003 ceph-mon[32262]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:15.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:15 smithi003 ceph-mon[32262]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:07:15.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:15 smithi003 ceph-mon[32262]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:15.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:15.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:15.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:15.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:15 smithi003 ceph-mon[32262]: Deploying cephadm binary to smithi163 2024-08-01T04:07:17.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.456+0000 7f6b62a00640 1 -- 172.21.15.3:0/1701973125 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6b6c1a2c60 con 0x7f6b500390c0 2024-08-01T04:07:17.458 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi163' with addr '172.21.15.163' 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.459+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6b500390c0 msgr2=0x7f6b5003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.459+0000 7f6b70c00640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6b500390c0 0x7f6b5003b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0098c0 tx=0x7f6b5c006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.460+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 msgr2=0x7f6b6c1a7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.460+0000 7f6b70c00640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c1a7660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009230 tx=0x7f6b54003d80 comp rx=0 tx=0).stop 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.460+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 shutdown_connections 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.460+0000 7f6b70c00640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6b500390c0 0x7f6b5003b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:17.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.460+0000 7f6b70c00640 1 --2- 172.21.15.3:0/1701973125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b6c10bb00 0x7f6b6c1a7660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.460+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 >> 172.21.15.3:0/1701973125 conn(0x7f6b6c06ca30 msgr2=0x7f6b6c070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.461+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 shutdown_connections 2024-08-01T04:07:17.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:17.461+0000 7f6b70c00640 1 -- 172.21.15.3:0/1701973125 wait complete. 2024-08-01T04:07:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:17 smithi003 ceph-mon[32262]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:18.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch host ls --format=json 2024-08-01T04:07:18.719 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:18.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:18.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:18 smithi003 ceph-mon[32262]: Added host smithi163 2024-08-01T04:07:18.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:18.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:18.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 -- 172.21.15.3:0/1231884420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 msgr2=0x7f9ccc101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 --2- 172.21.15.3:0/1231884420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc101710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009c20 tx=0x7f9cbc02aa90 comp rx=0 tx=0).stop 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 -- 172.21.15.3:0/1231884420 shutdown_connections 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 --2- 172.21.15.3:0/1231884420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc101710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 -- 172.21.15.3:0/1231884420 >> 172.21.15.3:0/1231884420 conn(0x7f9ccc0fcdd0 msgr2=0x7f9ccc0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 -- 172.21.15.3:0/1231884420 shutdown_connections 2024-08-01T04:07:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.310+0000 7f9cd3000640 1 -- 172.21.15.3:0/1231884420 wait complete. 2024-08-01T04:07:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.311+0000 7f9cd3000640 1 Processor -- start 2024-08-01T04:07:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.312+0000 7f9cd3000640 1 -- start start 2024-08-01T04:07:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.312+0000 7f9cd3000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:19.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.312+0000 7f9cd3000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10b780 con 0x7f9ccc101330 2024-08-01T04:07:19.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.312+0000 7f9cd1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:19.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.312+0000 7f9cd1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34200/0 (socket says 172.21.15.3:34200) 2024-08-01T04:07:19.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.312+0000 7f9cd1c00640 1 -- 172.21.15.3:0/1079469429 learned_addr learned my addr 172.21.15.3:0/1079469429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.313+0000 7f9cd1c00640 1 -- 172.21.15.3:0/1079469429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc1993b0 con 0x7f9ccc101330 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.313+0000 7f9cd1c00640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc198e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009d40 tx=0x7f9cbc02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.313+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cbc047070 con 0x7f9ccc101330 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.313+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cbc003e40 con 0x7f9ccc101330 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.313+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9cbc033050 con 0x7f9ccc101330 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.313+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc199640 con 0x7f9ccc101330 2024-08-01T04:07:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.314+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc199aa0 con 0x7f9ccc101330 2024-08-01T04:07:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.315+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9cbc04a070 con 0x7f9ccc101330 2024-08-01T04:07:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.315+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc102810 con 0x7f9ccc101330 2024-08-01T04:07:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.316+0000 7f9cca000640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9cb0039060 0x7f9cb003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.316+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9cbc03e030 con 0x7f9ccc101330 2024-08-01T04:07:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.316+0000 7f9cd1200640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9cb0039060 0x7f9cb003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:19.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.317+0000 7f9cd1200640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9cb0039060 0x7f9cb003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098c0 tx=0x7f9cb8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.321+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9cbc048260 con 0x7f9ccc101330 2024-08-01T04:07:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.525+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ccc19a230 con 0x7f9cb0039060 2024-08-01T04:07:19.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.529+0000 7f9cca000640 1 -- 172.21.15.3:0/1079469429 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f9ccc19a230 con 0x7f9cb0039060 2024-08-01T04:07:19.531 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:07:19.531 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}] 2024-08-01T04:07:19.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9cb0039060 msgr2=0x7f9cb003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:19.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9cb0039060 0x7f9cb003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098c0 tx=0x7f9cb8006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:19.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 msgr2=0x7f9ccc198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:19.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc198e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009d40 tx=0x7f9cbc02fc10 comp rx=0 tx=0).stop 2024-08-01T04:07:19.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 shutdown_connections 2024-08-01T04:07:19.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9cb0039060 0x7f9cb003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:19.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 --2- 172.21.15.3:0/1079469429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ccc101330 0x7f9ccc198e70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:19.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.532+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 >> 172.21.15.3:0/1079469429 conn(0x7f9ccc0fcdd0 msgr2=0x7f9ccc109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:19.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.533+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 shutdown_connections 2024-08-01T04:07:19.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:19.533+0000 7f9cd3000640 1 -- 172.21.15.3:0/1079469429 wait complete. 2024-08-01T04:07:20.168 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:19 smithi003 ceph-mon[32262]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:20.168 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:20.230 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-01T04:07:20.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd crush tunables default 2024-08-01T04:07:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:21.122 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:20 smithi003 ceph-mon[32262]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.262+0000 7fb12f400640 1 -- 172.21.15.3:0/1472675917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120031760 con 0x7fb13010dd70 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 -- 172.21.15.3:0/1472675917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 msgr2=0x7fb13010e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 --2- 172.21.15.3:0/1472675917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb13010e150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb120009880 tx=0x7fb12002a5d0 comp rx=0 tx=0).stop 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 -- 172.21.15.3:0/1472675917 shutdown_connections 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 --2- 172.21.15.3:0/1472675917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb13010e150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 -- 172.21.15.3:0/1472675917 >> 172.21.15.3:0/1472675917 conn(0x7fb13006ca30 msgr2=0x7fb13006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:21.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 -- 172.21.15.3:0/1472675917 shutdown_connections 2024-08-01T04:07:21.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.263+0000 7fb137000640 1 -- 172.21.15.3:0/1472675917 wait complete. 2024-08-01T04:07:21.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.264+0000 7fb137000640 1 Processor -- start 2024-08-01T04:07:21.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.264+0000 7fb137000640 1 -- start start 2024-08-01T04:07:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.265+0000 7fb137000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb1301a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.265+0000 7fb137000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb130112040 con 0x7fb13010dd70 2024-08-01T04:07:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.265+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb1301a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.265+0000 7fb12fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb1301a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34208/0 (socket says 172.21.15.3:34208) 2024-08-01T04:07:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.265+0000 7fb12fe00640 1 -- 172.21.15.3:0/3947050338 learned_addr learned my addr 172.21.15.3:0/3947050338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.265+0000 7fb12fe00640 1 -- 172.21.15.3:0/3947050338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1301a7ad0 con 0x7fb13010dd70 2024-08-01T04:07:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.266+0000 7fb12fe00640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb1301a7590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb120009230 tx=0x7fb120003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.266+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb12003f070 con 0x7fb13010dd70 2024-08-01T04:07:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.266+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12003bc80 con 0x7fb13010dd70 2024-08-01T04:07:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.266+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120035030 con 0x7fb13010dd70 2024-08-01T04:07:21.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.266+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1301a7dc0 con 0x7fb13010dd70 2024-08-01T04:07:21.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.266+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a1830 con 0x7fb13010dd70 2024-08-01T04:07:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.267+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb120042070 con 0x7fb13010dd70 2024-08-01T04:07:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.267+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb130109400 con 0x7fb13010dd70 2024-08-01T04:07:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.268+0000 7fb12d600640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb1100390b0 0x7fb11003b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.268+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb12006c960 con 0x7fb13010dd70 2024-08-01T04:07:21.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.268+0000 7fb127e00640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb1100390b0 0x7fb11003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:21.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.269+0000 7fb127e00640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb1100390b0 0x7fb11003b570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb1180098c0 tx=0x7fb118006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.273+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb120042360 con 0x7fb13010dd70 2024-08-01T04:07:21.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.472+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb1301a2b90 con 0x7fb13010dd70 2024-08-01T04:07:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.867+0000 7fb12d600640 1 -- 172.21.15.3:0/3947050338 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb1200412c0 con 0x7fb13010dd70 2024-08-01T04:07:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-08-01T04:07:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb1100390b0 msgr2=0x7fb11003b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb1100390b0 0x7fb11003b570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb1180098c0 tx=0x7fb118006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 msgr2=0x7fb1301a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb1301a7590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb120009230 tx=0x7fb120003d40 comp rx=0 tx=0).stop 2024-08-01T04:07:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 shutdown_connections 2024-08-01T04:07:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb1100390b0 0x7fb11003b570 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 --2- 172.21.15.3:0/3947050338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb13010dd70 0x7fb1301a7590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.871+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 >> 172.21.15.3:0/3947050338 conn(0x7fb13006ca30 msgr2=0x7fb130071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.872+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 shutdown_connections 2024-08-01T04:07:21.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:21.872+0000 7fb137000640 1 -- 172.21.15.3:0/3947050338 wait complete. 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3947050338' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:22.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:22.663 INFO:tasks.cephadm:Adding mon.a on smithi003 2024-08-01T04:07:22.663 INFO:tasks.cephadm:Adding mon.b on smithi160 2024-08-01T04:07:22.663 INFO:tasks.cephadm:Adding mon.c on smithi163 2024-08-01T04:07:22.663 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi160:172.21.15.160=b;smithi163:172.21.15.163=c' 2024-08-01T04:07:22.910 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:22 smithi003 ceph-mon[32262]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:07:23.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:22 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3947050338' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-01T04:07:23.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:22 smithi003 ceph-mon[32262]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:23.320 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.317+0000 7f4870e7e640 1 -- 172.21.15.163:0/3092639316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868030d20 con 0x7f486c100400 2024-08-01T04:07:23.320 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 -- 172.21.15.163:0/3092639316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 msgr2=0x7f486c1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:23.320 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 --2- 172.21.15.163:0/3092639316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c1007e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4868009810 tx=0x7f486802a540 comp rx=0 tx=0).stop 2024-08-01T04:07:23.320 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 -- 172.21.15.163:0/3092639316 shutdown_connections 2024-08-01T04:07:23.320 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 --2- 172.21.15.163:0/3092639316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c1007e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:23.321 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 -- 172.21.15.163:0/3092639316 >> 172.21.15.163:0/3092639316 conn(0x7f486c077040 msgr2=0x7f486c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:23.321 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 -- 172.21.15.163:0/3092639316 shutdown_connections 2024-08-01T04:07:23.321 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 -- 172.21.15.163:0/3092639316 wait complete. 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.318+0000 7f48740e4640 1 Processor -- start 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.319+0000 7f48740e4640 1 -- start start 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.319+0000 7f48740e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.319+0000 7f48740e4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f486c109840 con 0x7f486c100400 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.319+0000 7f4871e80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.319+0000 7f4871e80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.163:59708/0 (socket says 172.21.15.163:59708) 2024-08-01T04:07:23.322 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.319+0000 7f4871e80640 1 -- 172.21.15.163:0/3194486380 learned_addr learned my addr 172.21.15.163:0/3194486380 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:07:23.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.320+0000 7f4871e80640 1 -- 172.21.15.163:0/3194486380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c19adb0 con 0x7f486c100400 2024-08-01T04:07:23.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.320+0000 7f4871e80640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c19a870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48680045e0 tx=0x7f4868004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:23.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.321+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868030e70 con 0x7f486c100400 2024-08-01T04:07:23.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.321+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c19b0a0 con 0x7f486c100400 2024-08-01T04:07:23.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.321+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c194a30 con 0x7f486c100400 2024-08-01T04:07:23.323 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.321+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48680318e0 con 0x7f486c100400 2024-08-01T04:07:23.324 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.321+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868049060 con 0x7f486c100400 2024-08-01T04:07:23.324 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.322+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48500052c0 con 0x7f486c100400 2024-08-01T04:07:23.326 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.323+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4868039040 con 0x7f486c100400 2024-08-01T04:07:23.326 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.323+0000 7f4862ffd640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f485803b2c0 0x7f485803d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:23.326 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.323+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f486806d160 con 0x7f486c100400 2024-08-01T04:07:23.326 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.324+0000 7f487167f640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f485803b2c0 0x7f485803d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:23.327 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.325+0000 7f487167f640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f485803b2c0 0x7f485803d780 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f485c0098c0 tx=0x7f485c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:23.340 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.338+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4868040030 con 0x7f486c100400 2024-08-01T04:07:23.526 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.524+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi160:172.21.15.160=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}) -- 0x7f4850000cb0 con 0x7f485803b2c0 2024-08-01T04:07:23.540 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.538+0000 7f4862ffd640 1 -- 172.21.15.163:0/3194486380 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4850000cb0 con 0x7f485803b2c0 2024-08-01T04:07:23.541 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mon update... 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f485803b2c0 msgr2=0x7f485803d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f485803b2c0 0x7f485803d780 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f485c0098c0 tx=0x7f485c006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 msgr2=0x7f486c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c19a870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48680045e0 tx=0x7f4868004610 comp rx=0 tx=0).stop 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 shutdown_connections 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f485803b2c0 0x7f485803d780 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 --2- 172.21.15.163:0/3194486380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f486c100400 0x7f486c19a870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 >> 172.21.15.163:0/3194486380 conn(0x7f486c077040 msgr2=0x7f486c102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 shutdown_connections 2024-08-01T04:07:23.543 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:23.540+0000 7f48740e4640 1 -- 172.21.15.163:0/3194486380 wait complete. 2024-08-01T04:07:24.146 DEBUG:teuthology.orchestra.run.smithi160:mon.b> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.b.service 2024-08-01T04:07:24.148 DEBUG:teuthology.orchestra.run.smithi163:mon.c> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.c.service 2024-08-01T04:07:24.151 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-01T04:07:24.151 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph mon dump -f json 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:24.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:24.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:24.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:24.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:24.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:24 smithi003 ceph-mon[32262]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi160:172.21.15.160=b;smithi163:172.21.15.163=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:24 smithi003 ceph-mon[32262]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi160:172.21.15.160=b;smithi163:172.21.15.163=c;count:3 2024-08-01T04:07:25.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:24 smithi003 ceph-mon[32262]: Deploying daemon mon.c on smithi163 2024-08-01T04:07:26.021 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:07:26.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:25 smithi003 ceph-mon[32262]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:26.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 -- 172.21.15.163:0/2138050274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 msgr2=0x7f61cc1008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:26.659 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 --2- 172.21.15.163:0/2138050274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc1008f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009880 tx=0x7f61bc02a5d0 comp rx=0 tx=0).stop 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 -- 172.21.15.163:0/2138050274 shutdown_connections 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 --2- 172.21.15.163:0/2138050274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc1008f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 -- 172.21.15.163:0/2138050274 >> 172.21.15.163:0/2138050274 conn(0x7f61cc0fbfb0 msgr2=0x7f61cc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 -- 172.21.15.163:0/2138050274 shutdown_connections 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.655+0000 7f61d097b640 1 -- 172.21.15.163:0/2138050274 wait complete. 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.656+0000 7f61d097b640 1 Processor -- start 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.656+0000 7f61d097b640 1 -- start start 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.657+0000 7f61d097b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:26.660 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.657+0000 7f61d097b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10a960 con 0x7f61cc100510 2024-08-01T04:07:26.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.657+0000 7f61ca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:26.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.657+0000 7f61ca59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.163:59718/0 (socket says 172.21.15.163:59718) 2024-08-01T04:07:26.661 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.657+0000 7f61ca59c640 1 -- 172.21.15.163:0/706543049 learned_addr learned my addr 172.21.15.163:0/706543049 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.658+0000 7f61ca59c640 1 -- 172.21.15.163:0/706543049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc198300 con 0x7f61cc100510 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.658+0000 7f61ca59c640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc197dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009230 tx=0x7f61bc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.658+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61bc03f070 con 0x7f61cc100510 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.658+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc1985f0 con 0x7f61cc100510 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.659+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc198ab0 con 0x7f61cc100510 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.659+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61bc03bc80 con 0x7f61cc100510 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.659+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61bc049060 con 0x7f61cc100510 2024-08-01T04:07:26.662 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.659+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac0052c0 con 0x7f61cc100510 2024-08-01T04:07:26.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.660+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f61bc042070 con 0x7f61cc100510 2024-08-01T04:07:26.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.660+0000 7f61c37fe640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f61b8039060 0x7f61b803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:26.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.660+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f61bc06cca0 con 0x7f61cc100510 2024-08-01T04:07:26.663 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.661+0000 7f61c3fff640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f61b8039060 0x7f61b803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:26.664 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.662+0000 7f61c3fff640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f61b8039060 0x7f61b803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61b40098c0 tx=0x7f61b4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:26.676 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.674+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f61bc036260 con 0x7f61cc100510 2024-08-01T04:07:26.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.932+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f61ac005880 con 0x7f61cc100510 2024-08-01T04:07:26.936 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.933+0000 7f61c37fe640 1 -- 172.21.15.163:0/706543049 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f61bc044d30 con 0x7f61cc100510 2024-08-01T04:07:26.936 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:07:26.936 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","modified":"2024-08-01T04:05:42.092880Z","created":"2024-08-01T04:05:42.092880Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-01T04:07:26.937 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-08-01T04:07:26.937 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f61b8039060 msgr2=0x7f61b803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f61b8039060 0x7f61b803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f61b40098c0 tx=0x7f61b4006db0 comp rx=0 tx=0).stop 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 msgr2=0x7f61cc197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc197dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009230 tx=0x7f61bc003d40 comp rx=0 tx=0).stop 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 shutdown_connections 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f61b8039060 0x7f61b803b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 --2- 172.21.15.163:0/706543049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f61cc100510 0x7f61cc197dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 >> 172.21.15.163:0/706543049 conn(0x7f61cc0fbfb0 msgr2=0x7f61cc108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 shutdown_connections 2024-08-01T04:07:26.938 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:26.935+0000 7f61d097b640 1 -- 172.21.15.163:0/706543049 wait complete. 2024-08-01T04:07:28.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:27 smithi163 systemd[1]: Started Ceph mon.c for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:07:28.485 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-01T04:07:28.485 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph mon dump -f json 2024-08-01T04:07:28.734 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:07:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:07:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: mon.a calling monitor election 2024-08-01T04:07:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: mon.c calling monitor election 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: monmap epoch 2 2024-08-01T04:07:33.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: last_changed 2024-08-01T04:07:27.843321+0000 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: min_mon_release 19 (squid) 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: election_strategy: 1 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: fsmap 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: mgrmap e13: x(active, since 44s) 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: overall HEALTH_OK 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:33.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:33.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:33.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:33.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: mon.a calling monitor election 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: mon.c calling monitor election 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:33.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: monmap epoch 2 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: last_changed 2024-08-01T04:07:27.843321+0000 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: min_mon_release 19 (squid) 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: election_strategy: 1 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: fsmap 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: mgrmap e13: x(active, since 44s) 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: overall HEALTH_OK 2024-08-01T04:07:33.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:33.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:33.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:33.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:33.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:33.595 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/386039173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4100510 msgr2=0x7fc68c005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/386039173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4100510 0x7fc68c005580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc694012c60 tx=0x7fc69402c620 comp rx=0 tx=0).stop 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/386039173 shutdown_connections 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/386039173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4100510 0x7fc68c005580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/386039173 >> 172.21.15.163:0/386039173 conn(0x7fc6a40fbfb0 msgr2=0x7fc6a40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/386039173 shutdown_connections 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.592+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/386039173 wait complete. 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.593+0000 7fc6aa4ea640 1 Processor -- start 2024-08-01T04:07:33.596 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.593+0000 7fc6aa4ea640 1 -- start start 2024-08-01T04:07:33.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6aa4ea640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:33.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6aa4ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4109d10 0x7fc6a410a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:33.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6aa4ea640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41083c0 con 0x7fc6a4109d10 2024-08-01T04:07:33.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6aa4ea640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a4108240 con 0x7fc6a4100510 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34550/0 (socket says 172.21.15.163:34550) 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 -- 172.21.15.163:0/3015960594 learned_addr learned my addr 172.21.15.163:0/3015960594 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 -- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 msgr2=0x7fc6a41097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 -- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 msgr2=0x7fc6a41097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a3fff640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.594+0000 7fc6a37fe640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4109d10 0x7fc6a410a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.595+0000 7fc6a37fe640 1 -- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 msgr2=0x7fc6a41097d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:07:33.598 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.595+0000 7fc6a37fe640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:33.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.595+0000 7fc6a37fe640 1 -- 172.21.15.163:0/3015960594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a410b6c0 con 0x7fc6a4109d10 2024-08-01T04:07:33.599 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.596+0000 7fc6a37fe640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4109d10 0x7fc6a410a170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc69800b400 tx=0x7fc69800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.596+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc698011630 con 0x7fc6a4109d10 2024-08-01T04:07:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.596+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a41a3220 con 0x7fc6a4109d10 2024-08-01T04:07:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.596+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a41a37b0 con 0x7fc6a4109d10 2024-08-01T04:07:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.596+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc698011cb0 con 0x7fc6a4109d10 2024-08-01T04:07:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.596+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc698019da0 con 0x7fc6a4109d10 2024-08-01T04:07:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.597+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c002af0 con 0x7fc6a4109d10 2024-08-01T04:07:33.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.598+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc698018020 con 0x7fc6a4109d10 2024-08-01T04:07:33.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.598+0000 7fc6a17fa640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc690038e00 0x7fc69003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:33.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.598+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc69804b740 con 0x7fc6a4109d10 2024-08-01T04:07:33.601 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.599+0000 7fc6a3fff640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc690038e00 0x7fc69003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:33.602 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.600+0000 7fc6a3fff640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc690038e00 0x7fc69003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc694035040 tx=0x7fc694033660 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:33.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.612+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc69801f030 con 0x7fc6a4109d10 2024-08-01T04:07:33.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.846+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc68c002de0 con 0x7fc6a4109d10 2024-08-01T04:07:33.848 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.846+0000 7fc6a17fa640 1 -- 172.21.15.163:0/3015960594 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fc69800f780 con 0x7fc6a4109d10 2024-08-01T04:07:33.849 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:07:33.849 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","modified":"2024-08-01T04:07:27.843321Z","created":"2024-08-01T04:05:42.092880Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-01T04:07:33.849 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 2 2024-08-01T04:07:33.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc690038e00 msgr2=0x7fc69003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:33.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc690038e00 0x7fc69003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc694035040 tx=0x7fc694033660 comp rx=0 tx=0).stop 2024-08-01T04:07:33.851 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4109d10 msgr2=0x7fc6a410a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4109d10 0x7fc6a410a170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc69800b400 tx=0x7fc69800b8c0 comp rx=0 tx=0).stop 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 shutdown_connections 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc690038e00 0x7fc69003b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6a4109d10 0x7fc6a410a170 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 --2- 172.21.15.163:0/3015960594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6a4100510 0x7fc6a41097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 >> 172.21.15.163:0/3015960594 conn(0x7fc6a40fbfb0 msgr2=0x7fc6a4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 shutdown_connections 2024-08-01T04:07:33.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:33.849+0000 7fc6aa4ea640 1 -- 172.21.15.163:0/3015960594 wait complete. 2024-08-01T04:07:34.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:33 smithi163 ceph-mon[37383]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:34.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:33 smithi163 ceph-mon[37383]: Deploying daemon mon.b on smithi160 2024-08-01T04:07:34.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:34.142 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:33 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/3015960594' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-01T04:07:34.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:33 smithi003 ceph-mon[32262]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:34.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:33 smithi003 ceph-mon[32262]: Deploying daemon mon.b on smithi160 2024-08-01T04:07:34.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:34.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:33 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/3015960594' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-01T04:07:35.036 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:07:34 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:07:34.844+0000 7f3ab1a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-01T04:07:35.437 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-01T04:07:35.438 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph mon dump -f json 2024-08-01T04:07:35.691 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:07:36.068 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:35 smithi163 ceph-mon[37383]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:36.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:35 smithi003 ceph-mon[32262]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 -- 172.21.15.163:0/180031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 msgr2=0x7f1d9c102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 --2- 172.21.15.163:0/180031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c102950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009880 tx=0x7f1d9802a670 comp rx=0 tx=0).stop 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 -- 172.21.15.163:0/180031991 shutdown_connections 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 --2- 172.21.15.163:0/180031991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c102950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 -- 172.21.15.163:0/180031991 >> 172.21.15.163:0/180031991 conn(0x7f1d9c0fbfb0 msgr2=0x7f1d9c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 -- 172.21.15.163:0/180031991 shutdown_connections 2024-08-01T04:07:36.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 -- 172.21.15.163:0/180031991 wait complete. 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 Processor -- start 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.358+0000 7f1da42d6640 1 -- start start 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da42d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da42d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da42d6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c10bae0 con 0x7f1d9c102570 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da42d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c10b960 con 0x7f1d9c197710 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34558/0 (socket says 172.21.15.163:34558) 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 -- 172.21.15.163:0/1738093813 learned_addr learned my addr 172.21.15.163:0/1738093813 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:07:36.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 -- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 msgr2=0x7f1d9c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-01T04:07:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 -- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 msgr2=0x7f1d9c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-01T04:07:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-01T04:07:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da1871640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:07:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.359+0000 7f1da2072640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.360+0000 7f1da2072640 1 -- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 msgr2=0x7f1d9c19ba50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.360+0000 7f1da2072640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.360+0000 7f1da2072640 1 -- 172.21.15.163:0/1738093813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c197d30 con 0x7f1d9c102570 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.360+0000 7f1da2072640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c1971d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005ab0 tx=0x7f1d98032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.361+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1d98035070 con 0x7f1d9c102570 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.361+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c19bf90 con 0x7f1d9c102570 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.361+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c19c470 con 0x7f1d9c102570 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.361+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98032e40 con 0x7f1d9c102570 2024-08-01T04:07:36.366 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.361+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1d9803b800 con 0x7f1d9c102570 2024-08-01T04:07:36.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.362+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d80005290 con 0x7f1d9c102570 2024-08-01T04:07:36.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.363+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1d98042030 con 0x7f1d9c102570 2024-08-01T04:07:36.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.363+0000 7f1d937fe640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1d88038e00 0x7f1d8803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:36.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.363+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d98040070 con 0x7f1d9c102570 2024-08-01T04:07:36.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.363+0000 7f1da1871640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1d88038e00 0x7f1d8803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:36.367 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.364+0000 7f1da1871640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1d88038e00 0x7f1d8803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009100 tx=0x7f1d8c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:36.378 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.377+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d98004e20 con 0x7f1d9c102570 2024-08-01T04:07:36.597 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.595+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9803b9a0 con 0x7f1d9c102570 2024-08-01T04:07:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:36.607+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1d80005580 con 0x7f1d9c102570 2024-08-01T04:07:36.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:36 smithi160 systemd[1]: Started Ceph mon.b for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:07:41.616 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.614+0000 7f1d937fe640 1 -- 172.21.15.163:0/1738093813 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f1d98044c80 con 0x7f1d9c102570 2024-08-01T04:07:41.616 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:07:41.616 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":3,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","modified":"2024-08-01T04:07:36.594390Z","created":"2024-08-01T04:05:42.092880Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-01T04:07:41.616 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 3 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1d88038e00 msgr2=0x7f1d8803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1d88038e00 0x7f1d8803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009100 tx=0x7f1d8c008040 comp rx=0 tx=0).stop 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 msgr2=0x7f1d9c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c1971d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005ab0 tx=0x7f1d98032c50 comp rx=0 tx=0).stop 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 shutdown_connections 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1d88038e00 0x7f1d8803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d9c197710 0x7f1d9c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 --2- 172.21.15.163:0/1738093813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c102570 0x7f1d9c1971d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 >> 172.21.15.163:0/1738093813 conn(0x7f1d9c0fbfb0 msgr2=0x7f1d9c105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 shutdown_connections 2024-08-01T04:07:41.619 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:41.617+0000 7f1da42d6640 1 -- 172.21.15.163:0/1738093813 wait complete. 2024-08-01T04:07:42.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:42.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: mon.c calling monitor election 2024-08-01T04:07:42.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:07:42.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: mon.a calling monitor election 2024-08-01T04:07:42.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/1738093813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-01T04:07:42.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: monmap epoch 3 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: last_changed 2024-08-01T04:07:36.594390+0000 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: min_mon_release 19 (squid) 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: election_strategy: 1 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: fsmap 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: mgrmap e13: x(active, since 53s) 2024-08-01T04:07:42.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: overall HEALTH_OK 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: mon.c calling monitor election 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:07:42.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: mon.a calling monitor election 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/1738093813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-01T04:07:42.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: monmap epoch 3 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: last_changed 2024-08-01T04:07:36.594390+0000 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: min_mon_release 19 (squid) 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: election_strategy: 1 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: fsmap 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: mgrmap e13: x(active, since 53s) 2024-08-01T04:07:42.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: overall HEALTH_OK 2024-08-01T04:07:42.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:42.656 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-01T04:07:42.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph config generate-minimal-conf 2024-08-01T04:07:43.043 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:42 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:43.043 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:42 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:42 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:43.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:42 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.125 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: mon.c calling monitor election 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: mon.a calling monitor election 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/1738093813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-01T04:07:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: monmap epoch 3 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: last_changed 2024-08-01T04:07:36.594390+0000 2024-08-01T04:07:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: min_mon_release 19 (squid) 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: election_strategy: 1 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: fsmap 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: mgrmap e13: x(active, since 53s) 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: overall HEALTH_OK 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:43.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:43.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:43.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:43.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 -- 172.21.15.3:0/3437581556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18109570 msgr2=0x7f9b18109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:43.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 --2- 172.21.15.3:0/3437581556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18109570 0x7f9b18109950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009880 tx=0x7f9b0802a5d0 comp rx=0 tx=0).stop 2024-08-01T04:07:43.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 -- 172.21.15.3:0/3437581556 shutdown_connections 2024-08-01T04:07:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 --2- 172.21.15.3:0/3437581556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18109570 0x7f9b18109950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 -- 172.21.15.3:0/3437581556 >> 172.21.15.3:0/3437581556 conn(0x7f9b180ff180 msgr2=0x7f9b181015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 -- 172.21.15.3:0/3437581556 shutdown_connections 2024-08-01T04:07:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.953+0000 7f9b1d800640 1 -- 172.21.15.3:0/3437581556 wait complete. 2024-08-01T04:07:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.954+0000 7f9b1d800640 1 Processor -- start 2024-08-01T04:07:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.954+0000 7f9b1d800640 1 -- start start 2024-08-01T04:07:43.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b1d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 0x7f9b18199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:43.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b1d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18199870 0x7f9b1819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:43.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b1d800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b1819e0f0 0x7f9b1819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b1d800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810dba0 con 0x7f9b18199870 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b1d800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810da20 con 0x7f9b1819e0f0 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b1d800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1810dd20 con 0x7f9b18109570 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b16a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 0x7f9b18199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b17400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b1819e0f0 0x7f9b1819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b16000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18199870 0x7f9b1819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.955+0000 7f9b16a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 0x7f9b18199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54886/0 (socket says 172.21.15.3:54886) 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 -- 172.21.15.3:0/2369530253 learned_addr learned my addr 172.21.15.3:0/2369530253 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 -- 172.21.15.3:0/2369530253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b1819e0f0 msgr2=0x7f9b1819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b1819e0f0 0x7f9b1819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 -- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18199870 msgr2=0x7f9b1819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18199870 0x7f9b1819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:43.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 -- 172.21.15.3:0/2369530253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b181a5e60 con 0x7f9b18109570 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16000640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18199870 0x7f9b1819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.956+0000 7f9b16a00640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 0x7f9b18199330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9b08006010 tx=0x7f9b08003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.957+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0803f070 con 0x7f9b18109570 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.957+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b0803b460 con 0x7f9b18109570 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.957+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b181a6030 con 0x7f9b18109570 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.957+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08035030 con 0x7f9b18109570 2024-08-01T04:07:43.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.957+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b181a6530 con 0x7f9b18109570 2024-08-01T04:07:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.958+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9b08042070 con 0x7f9b18109570 2024-08-01T04:07:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.959+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b181099d0 con 0x7f9b18109570 2024-08-01T04:07:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.959+0000 7f9b0f400640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9afc0394f0 0x7f9afc03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.959+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9b0806c8f0 con 0x7f9b18109570 2024-08-01T04:07:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.959+0000 7f9b16000640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9afc0394f0 0x7f9afc03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.960+0000 7f9b16000640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9afc0394f0 0x7f9afc03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009e20 tx=0x7f9b04009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:43.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:43.964+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9b080303a0 con 0x7f9b18109570 2024-08-01T04:07:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.165+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9b181a67f0 con 0x7f9b18109570 2024-08-01T04:07:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.165+0000 7f9b0f400640 1 -- 172.21.15.3:0/2369530253 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f9b0802fdc0 con 0x7f9b18109570 2024-08-01T04:07:44.166 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:44.167 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-08-01T04:07:44.167 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:44.167 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2024-08-01T04:07:44.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9afc0394f0 msgr2=0x7f9afc03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:44.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9afc0394f0 0x7f9afc03b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009e20 tx=0x7f9b04009190 comp rx=0 tx=0).stop 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 msgr2=0x7f9b18199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 0x7f9b18199330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9b08006010 tx=0x7f9b08003c00 comp rx=0 tx=0).stop 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 shutdown_connections 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9afc0394f0 0x7f9afc03b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b1819e0f0 0x7f9b1819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b18199870 0x7f9b1819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 --2- 172.21.15.3:0/2369530253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b18109570 0x7f9b18199330 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 >> 172.21.15.3:0/2369530253 conn(0x7f9b180ff180 msgr2=0x7f9b180ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 shutdown_connections 2024-08-01T04:07:44.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:44.168+0000 7f9b1d800640 1 -- 172.21.15.3:0/2369530253 wait complete. 2024-08-01T04:07:44.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: mon.b calling monitor election 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: mon.b calling monitor election 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: mon.c calling monitor election 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: mon.a calling monitor election 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: monmap epoch 3 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: last_changed 2024-08-01T04:07:36.594390+0000 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:44.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: min_mon_release 19 (squid) 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: election_strategy: 1 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: fsmap 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: mgrmap e13: x(active, since 55s) 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: overall HEALTH_OK 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.405 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.405 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:44 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2369530253' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: mon.b calling monitor election 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: mon.b calling monitor election 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: mon.c calling monitor election 2024-08-01T04:07:44.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: mon.a calling monitor election 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: monmap epoch 3 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: last_changed 2024-08-01T04:07:36.594390+0000 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: min_mon_release 19 (squid) 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: election_strategy: 1 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: fsmap 2024-08-01T04:07:44.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: mgrmap e13: x(active, since 55s) 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: overall HEALTH_OK 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:44 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2369530253' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: mon.b calling monitor election 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: mon.b calling monitor election 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: mon.c calling monitor election 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: mon.a calling monitor election 2024-08-01T04:07:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: monmap epoch 3 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: last_changed 2024-08-01T04:07:36.594390+0000 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: created 2024-08-01T04:05:42.092880+0000 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: min_mon_release 19 (squid) 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: election_strategy: 1 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: 1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.c 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: fsmap 2024-08-01T04:07:44.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: osdmap e4: 0 total, 0 up, 0 in 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: mgrmap e13: x(active, since 55s) 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: overall HEALTH_OK 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:44.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:44 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2369530253' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:45.133 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-01T04:07:45.134 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:07:45.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2024-08-01T04:07:45.218 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:07:45.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:45.330 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:07:45.330 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-08-01T04:07:45.358 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:07:45.358 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:45.423 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:07:45.423 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.conf 2024-08-01T04:07:45.451 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:07:45.451 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:07:45.516 INFO:tasks.cephadm:Adding mgr.x on smithi003 2024-08-01T04:07:45.516 INFO:tasks.cephadm:Adding mgr.y on smithi160 2024-08-01T04:07:45.516 INFO:tasks.cephadm:Adding mgr.z on smithi163 2024-08-01T04:07:45.517 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch apply mgr '3;smithi003=x;smithi160=y;smithi163=z' 2024-08-01T04:07:45.577 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:45.577 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:45.577 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:45.577 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.577 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: Reconfiguring mon.a (unknown last config time)... 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:45.578 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:45 smithi163 ceph-mon[37383]: Reconfiguring daemon mon.a on smithi003 2024-08-01T04:07:45.631 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:45.631 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:45.631 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:45.631 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.631 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: Reconfiguring mon.a (unknown last config time)... 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:45.632 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:45 smithi003 ceph-mon[32262]: Reconfiguring daemon mon.a on smithi003 2024-08-01T04:07:45.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:07:45.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:45.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: Reconfiguring mon.a (unknown last config time)... 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:45.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:45 smithi160 ceph-mon[37224]: Reconfiguring daemon mon.a on smithi003 2024-08-01T04:07:45.799 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:07:46.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 -- 172.21.15.163:0/24532252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 msgr2=0x7f3314100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:46.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 --2- 172.21.15.163:0/24532252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 0x7f3314100720 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3304009c20 tx=0x7f330402aa90 comp rx=0 tx=0).stop 2024-08-01T04:07:46.223 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 -- 172.21.15.163:0/24532252 shutdown_connections 2024-08-01T04:07:46.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 --2- 172.21.15.163:0/24532252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 0x7f3314100720 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 -- 172.21.15.163:0/24532252 >> 172.21.15.163:0/24532252 conn(0x7f33140fbad0 msgr2=0x7f33140fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:46.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 -- 172.21.15.163:0/24532252 shutdown_connections 2024-08-01T04:07:46.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.221+0000 7f33199da640 1 -- 172.21.15.163:0/24532252 wait complete. 2024-08-01T04:07:46.224 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.222+0000 7f33199da640 1 Processor -- start 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.222+0000 7f33199da640 1 -- start start 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33199da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 0x7f3314195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33199da640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314196170 0x7f331419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33199da640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 0x7f331419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33199da640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3314106410 con 0x7f3314100320 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33199da640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3314106290 con 0x7f3314196170 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33199da640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3314106590 con 0x7f33141965f0 2024-08-01T04:07:46.227 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 0x7f331419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 0x7f331419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53942/0 (socket says 172.21.15.163:53942) 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 -- 172.21.15.163:0/3480161596 learned_addr learned my addr 172.21.15.163:0/3480161596 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33127fc640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314196170 0x7f331419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f3312ffd640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 0x7f3314195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 -- 172.21.15.163:0/3480161596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314196170 msgr2=0x7f331419b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314196170 0x7f331419b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.228 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 -- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 msgr2=0x7f3314195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 0x7f3314195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.223+0000 7f33137fe640 1 -- 172.21.15.163:0/3480161596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331419dff0 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.224+0000 7f33137fe640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 0x7f331419d730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f330c00b450 tx=0x7f330c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.224+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c015020 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.224+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f330c011d70 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.224+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331419e2e0 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.224+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c00f340 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.224+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331419e850 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.225+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f330c018020 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.225+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331418f1d0 con 0x7f33141965f0 2024-08-01T04:07:46.229 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.225+0000 7f33189d8640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f32fc0394f0 0x7f32fc03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.225+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f330c04be40 con 0x7f33141965f0 2024-08-01T04:07:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.226+0000 7f3312ffd640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f32fc0394f0 0x7f32fc03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:46.230 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.226+0000 7f3312ffd640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f32fc0394f0 0x7f32fc03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3304000f80 tx=0x7f330403a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:46.233 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.230+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f330c01f020 con 0x7f33141965f0 2024-08-01T04:07:46.411 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.409+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi160=y;smithi163=z", "target": ["mon-mgr", ""]}) -- 0x7f33140623f0 con 0x7f32fc0394f0 2024-08-01T04:07:46.421 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.420+0000 7f33189d8640 1 -- 172.21.15.163:0/3480161596 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f33140623f0 con 0x7f32fc0394f0 2024-08-01T04:07:46.421 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mgr update... 2024-08-01T04:07:46.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f32fc0394f0 msgr2=0x7f32fc03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:46.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f32fc0394f0 0x7f32fc03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3304000f80 tx=0x7f330403a040 comp rx=0 tx=0).stop 2024-08-01T04:07:46.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 msgr2=0x7f331419d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:46.423 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 0x7f331419d730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f330c00b450 tx=0x7f330c00b910 comp rx=0 tx=0).stop 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 shutdown_connections 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f32fc0394f0 0x7f32fc03b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33141965f0 0x7f331419d730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3314196170 0x7f331419b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 --2- 172.21.15.163:0/3480161596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3314100320 0x7f3314195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 >> 172.21.15.163:0/3480161596 conn(0x7f33140fbad0 msgr2=0x7f33140fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 shutdown_connections 2024-08-01T04:07:46.424 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:07:46.422+0000 7f33199da640 1 -- 172.21.15.163:0/3480161596 wait complete. 2024-08-01T04:07:46.953 DEBUG:teuthology.orchestra.run.smithi160:mgr.y> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.y.service 2024-08-01T04:07:46.955 DEBUG:teuthology.orchestra.run.smithi163:mgr.z> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.z.service 2024-08-01T04:07:46.996 INFO:tasks.cephadm:Deploying OSDs... 2024-08-01T04:07:46.996 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:07:46.996 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-08-01T04:07:47.048 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-01T04:07:47.049 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-08-01 04:07:02.042680789 +0000 2024-08-01T04:07:47.091 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-08-01 04:01:55.146603786 +0000 2024-08-01T04:07:47.092 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-08-01 04:01:55.146603786 +0000 2024-08-01T04:07:47.092 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-08-01 04:01:55.146603786 +0000 2024-08-01T04:07:47.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-01T04:07:47.199 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-08-01T04:07:47.199 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-08-01T04:07:47.199 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000342809 s, 1.5 MB/s 2024-08-01T04:07:47.202 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-01T04:07:47.249 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-08-01T04:07:47.294 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-01T04:07:47.294 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-08-01 04:07:02.089680194 +0000 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-08-01 04:01:55.721596560 +0000 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-08-01 04:01:55.721596560 +0000 2024-08-01T04:07:47.295 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-08-01 04:01:55.721596560 +0000 2024-08-01T04:07:47.295 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-01T04:07:47.416 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-08-01T04:07:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-08-01T04:07:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000307931 s, 1.7 MB/s 2024-08-01T04:07:47.421 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-01T04:07:47.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:47.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi160=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:47.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: Saving service mgr spec with placement smithi003=x;smithi160=y;smithi163=z;count:3 2024-08-01T04:07:47.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: Reconfiguring mgr.x (unknown last config time)... 2024-08-01T04:07:47.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:47.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:47.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:47.448 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:47 smithi003 ceph-mon[32262]: Reconfiguring daemon mgr.x on smithi003 2024-08-01T04:07:47.469 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-08-01 04:07:02.135679613 +0000 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-08-01 04:01:56.367588442 +0000 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-08-01 04:01:56.367588442 +0000 2024-08-01T04:07:47.553 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-08-01 04:01:56.367588442 +0000 2024-08-01T04:07:47.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-01T04:07:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-08-01T04:07:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-08-01T04:07:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000264127 s, 1.9 MB/s 2024-08-01T04:07:47.659 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-01T04:07:47.698 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-08-01T04:07:47.785 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-01T04:07:47.785 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-08-01 04:07:02.181679032 +0000 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-08-01 04:01:56.961580978 +0000 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-08-01 04:01:56.961580978 +0000 2024-08-01T04:07:47.786 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-08-01 04:01:56.961580978 +0000 2024-08-01T04:07:47.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi160=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: Saving service mgr spec with placement smithi003=x;smithi160=y;smithi163=z;count:3 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: Reconfiguring mgr.x (unknown last config time)... 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:47.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:47.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:47.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:47 smithi163 ceph-mon[37383]: Reconfiguring daemon mgr.x on smithi003 2024-08-01T04:07:47.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi160=y;smithi163=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: Saving service mgr spec with placement smithi003=x;smithi160=y;smithi163=z;count:3 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: Reconfiguring mgr.x (unknown last config time)... 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:47.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:47 smithi160 ceph-mon[37224]: Reconfiguring daemon mgr.x on smithi003 2024-08-01T04:07:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-08-01T04:07:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-08-01T04:07:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000373329 s, 1.4 MB/s 2024-08-01T04:07:47.907 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-01T04:07:47.966 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-08-01T04:07:47.966 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-08-01T04:07:47.985 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-01T04:07:47.985 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-08-01 04:07:36.050030614 +0000 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-08-01 04:00:30.730534804 +0000 2024-08-01T04:07:48.042 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-08-01 04:00:30.730534804 +0000 2024-08-01T04:07:48.043 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-08-01 04:00:30.730534804 +0000 2024-08-01T04:07:48.043 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-01T04:07:48.107 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-08-01T04:07:48.108 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-08-01T04:07:48.108 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000254425 s, 2.0 MB/s 2024-08-01T04:07:48.109 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-01T04:07:48.166 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-08-01T04:07:48.222 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-01T04:07:48.222 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:48.222 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-01T04:07:48.222 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:48.222 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:48.223 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-08-01 04:07:36.050030614 +0000 2024-08-01T04:07:48.223 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-08-01 04:00:31.069529972 +0000 2024-08-01T04:07:48.223 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-08-01 04:00:31.069529972 +0000 2024-08-01T04:07:48.223 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-08-01 04:00:31.069529972 +0000 2024-08-01T04:07:48.223 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-01T04:07:48.286 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-08-01T04:07:48.286 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-08-01T04:07:48.286 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000220323 s, 2.3 MB/s 2024-08-01T04:07:48.287 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-01T04:07:48.343 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-08-01T04:07:48.398 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-01T04:07:48.398 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-08-01 04:07:36.050030614 +0000 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-08-01 04:00:31.403525212 +0000 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-08-01 04:00:31.403525212 +0000 2024-08-01T04:07:48.399 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-08-01 04:00:31.403525212 +0000 2024-08-01T04:07:48.399 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-01T04:07:48.461 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-08-01T04:07:48.461 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-08-01T04:07:48.461 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000217969 s, 2.3 MB/s 2024-08-01T04:07:48.462 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-01T04:07:48.519 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-08-01 04:07:36.051030600 +0000 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-08-01 04:00:31.721520680 +0000 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-08-01 04:00:31.721520680 +0000 2024-08-01T04:07:48.574 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-08-01 04:00:31.721520680 +0000 2024-08-01T04:07:48.575 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-01T04:07:48.636 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-08-01T04:07:48.636 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-08-01T04:07:48.636 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000193359 s, 2.6 MB/s 2024-08-01T04:07:48.638 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-01T04:07:48.692 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-08-01T04:07:48.692 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-08-01T04:07:48.711 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-01T04:07:48.711 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-08-01T04:07:48.769 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-08-01 04:07:27.019875683 +0000 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-08-01 04:00:30.733045573 +0000 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-08-01 04:00:30.733045573 +0000 2024-08-01T04:07:48.770 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-08-01 04:00:30.733045573 +0000 2024-08-01T04:07:48.770 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-01T04:07:48.836 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-08-01T04:07:48.836 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-08-01T04:07:48.836 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000208128 s, 2.5 MB/s 2024-08-01T04:07:48.837 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-01T04:07:48.895 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-08-01T04:07:48.951 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-01T04:07:48.951 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:48.951 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-01T04:07:48.951 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:48.952 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:48.952 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-08-01 04:07:27.020875661 +0000 2024-08-01T04:07:48.952 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-08-01 04:00:31.068038199 +0000 2024-08-01T04:07:48.952 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-08-01 04:00:31.068038199 +0000 2024-08-01T04:07:48.952 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-08-01 04:00:31.068038199 +0000 2024-08-01T04:07:48.952 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-01T04:07:49.016 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-08-01T04:07:49.016 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-08-01T04:07:49.016 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000211961 s, 2.4 MB/s 2024-08-01T04:07:49.018 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-01T04:07:49.075 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-08-01T04:07:49.130 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-08-01 04:07:27.020875661 +0000 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-08-01 04:00:31.402030847 +0000 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-08-01 04:00:31.402030847 +0000 2024-08-01T04:07:49.131 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-08-01 04:00:31.402030847 +0000 2024-08-01T04:07:49.131 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-01T04:07:49.194 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-08-01T04:07:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-08-01T04:07:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000274296 s, 1.9 MB/s 2024-08-01T04:07:49.196 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-01T04:07:49.252 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-08-01T04:07:49.306 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-01T04:07:49.309 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-01T04:07:49.309 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-01T04:07:49.310 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:07:49.310 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-08-01T04:07:49.310 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-08-01 04:07:27.020875661 +0000 2024-08-01T04:07:49.310 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-08-01 04:00:31.723023781 +0000 2024-08-01T04:07:49.310 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-08-01 04:00:31.723023781 +0000 2024-08-01T04:07:49.310 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-08-01 04:00:31.723023781 +0000 2024-08-01T04:07:49.310 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-01T04:07:49.369 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-08-01T04:07:49.369 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-08-01T04:07:49.369 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000210814 s, 2.4 MB/s 2024-08-01T04:07:49.370 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-01T04:07:49.426 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/vg_nvme/lv_4... 2024-08-01T04:07:49.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-01T04:07:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: Reconfiguring mon.b (monmap changed)... 2024-08-01T04:07:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:49.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:49.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:49.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:49 smithi003 ceph-mon[32262]: Reconfiguring daemon mon.b on smithi160 2024-08-01T04:07:49.843 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:49.843 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:49.843 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:49.843 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: Reconfiguring mon.b (monmap changed)... 2024-08-01T04:07:49.843 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:49.844 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:49.844 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:49.844 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:49 smithi160 ceph-mon[37224]: Reconfiguring daemon mon.b on smithi160 2024-08-01T04:07:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:50.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: Reconfiguring mon.b (monmap changed)... 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:49 smithi163 ceph-mon[37383]: Reconfiguring daemon mon.b on smithi160 2024-08-01T04:07:51.074 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: Reconfiguring mon.c (monmap changed)... 2024-08-01T04:07:51.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:51.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:51.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:51.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:50 smithi003 ceph-mon[32262]: Reconfiguring daemon mon.c on smithi163 2024-08-01T04:07:51.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: Reconfiguring mon.c (monmap changed)... 2024-08-01T04:07:51.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:51.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:51.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:51.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:50 smithi160 ceph-mon[37224]: Reconfiguring daemon mon.c on smithi163 2024-08-01T04:07:51.156 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.156 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.157 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: Reconfiguring mon.c (monmap changed)... 2024-08-01T04:07:51.157 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-01T04:07:51.157 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-01T04:07:51.157 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:51.157 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:50 smithi163 ceph-mon[37383]: Reconfiguring daemon mon.c on smithi163 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:51.951 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:51.952 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:51.952 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-01T04:07:51.952 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:51.952 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:52.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:52.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:52.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:53.233 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:52 smithi003 ceph-mon[32262]: Deploying daemon mgr.z on smithi163 2024-08-01T04:07:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:07:53.251 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:52 smithi163 ceph-mon[37383]: Deploying daemon mgr.z on smithi163 2024-08-01T04:07:53.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_4 2024-08-01T04:07:53.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:52 smithi160 ceph-mon[37224]: Deploying daemon mgr.z on smithi163 2024-08-01T04:07:53.546 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:53.484+0000 7f3226cde040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:07:53.788 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:07:53.935 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:53.643+0000 7f3226cde040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:07:53.935 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:53.710+0000 7f3226cde040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:07:54.226 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:54.226 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.226 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.226 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.226 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:54.227 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-01T04:07:54.227 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:54.227 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:54.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:54.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:53.956+0000 7f3226cde040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:54 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:54.033+0000 7f3226cde040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-01T04:07:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-01T04:07:54.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:54.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 -- 172.21.15.3:0/2288056080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 msgr2=0x7fe504105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 --2- 172.21.15.3:0/2288056080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe504105000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009860 tx=0x7fe4ec02a340 comp rx=0 tx=0).stop 2024-08-01T04:07:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 -- 172.21.15.3:0/2288056080 shutdown_connections 2024-08-01T04:07:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 --2- 172.21.15.3:0/2288056080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504105540 0x7fe50410bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 --2- 172.21.15.3:0/2288056080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe504105000 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 --2- 172.21.15.3:0/2288056080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe504103980 0x7fe504103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 -- 172.21.15.3:0/2288056080 >> 172.21.15.3:0/2288056080 conn(0x7fe5040ff110 msgr2=0x7fe504101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:07:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 -- 172.21.15.3:0/2288056080 shutdown_connections 2024-08-01T04:07:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.384+0000 7fe509000640 1 -- 172.21.15.3:0/2288056080 wait complete. 2024-08-01T04:07:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.385+0000 7fe509000640 1 Processor -- start 2024-08-01T04:07:54.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 -- start start 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 0x7fe504199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe50419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe504105540 0x7fe5041a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410f600 con 0x7fe504104b80 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410f480 con 0x7fe504105540 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe509000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe50410f780 con 0x7fe504103980 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe503400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 0x7fe504199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.386+0000 7fe503400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 0x7fe504199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:38592/0 (socket says 172.21.15.3:38592) 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 -- 172.21.15.3:0/4188087497 learned_addr learned my addr 172.21.15.3:0/4188087497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503e00640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe504105540 0x7fe5041a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe4faa00640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe50419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 -- 172.21.15.3:0/4188087497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe504105540 msgr2=0x7fe5041a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe504105540 0x7fe5041a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 -- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 msgr2=0x7fe50419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe50419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 -- 172.21.15.3:0/4188087497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041a1890 con 0x7fe504103980 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe4faa00640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe50419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.387+0000 7fe503400640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 0x7fe504199c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400e9c0 tx=0x7fe4f400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.388+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f400cd80 con 0x7fe504103980 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.388+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4f40035f0 con 0x7fe504103980 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.388+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4010690 con 0x7fe504103980 2024-08-01T04:07:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.388+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5041a1b80 con 0x7fe504103980 2024-08-01T04:07:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.388+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041a2070 con 0x7fe504103980 2024-08-01T04:07:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.389+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe4f4004020 con 0x7fe504103980 2024-08-01T04:07:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.389+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50410ada0 con 0x7fe504103980 2024-08-01T04:07:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.389+0000 7fe500c00640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe4e8039290 0x7fe4e803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:07:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.390+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe4f404c820 con 0x7fe504103980 2024-08-01T04:07:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.390+0000 7fe4faa00640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe4e8039290 0x7fe4e803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:07:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.390+0000 7fe4faa00640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe4e8039290 0x7fe4e803b750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec030550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:07:54.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.399+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe4f40071c0 con 0x7fe504103980 2024-08-01T04:07:54.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:54.605+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe504062360 con 0x7fe4e8039290 2024-08-01T04:07:55.133 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.015+0000 7f3226cde040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:07:55.133 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.131+0000 7f3226cde040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:07:55.133 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:54 smithi163 ceph-mon[37383]: Deploying daemon mgr.y on smithi160 2024-08-01T04:07:55.133 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:54 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:07:55.133 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:54 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:07:55.133 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:54 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:55.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:54 smithi160 ceph-mon[37224]: Deploying daemon mgr.y on smithi160 2024-08-01T04:07:55.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:54 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:07:55.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:54 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:07:55.201 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:54 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:55.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:54 smithi003 ceph-mon[32262]: Deploying daemon mgr.y on smithi160 2024-08-01T04:07:55.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:54 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:07:55.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:54 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:07:55.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:54 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:55.399 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.188+0000 7f3226cde040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:07:55.399 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.398+0000 7f3226cde040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:07:55.564 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 systemd[1]: Starting Ceph mgr.y for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:07:55.669 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.458+0000 7f3226cde040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:07:55.669 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.523+0000 7f3226cde040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:07:55.669 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:55.668+0000 7f3226cde040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:07:55.865 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 podman[38577]: 2024-08-01 04:07:55.562931967 +0000 UTC m=+0.109003101 container create 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-01T04:07:55.865 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 podman[38577]: 2024-08-01 04:07:55.471286798 +0000 UTC m=+0.017357925 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:07:55.865 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 podman[38577]: 2024-08-01 04:07:55.740242387 +0000 UTC m=+0.286313529 container init 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-01T04:07:55.865 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 podman[38577]: 2024-08-01 04:07:55.743088832 +0000 UTC m=+0.289159967 container start 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-01T04:07:55.865 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 ceph-mgr[38596]: -- 172.21.15.160:0/3295252343 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1c509b0e0 con 0x55b1c5078800 2024-08-01T04:07:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:55 smithi160 ceph-mon[37224]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:55 smithi160 ceph-mon[37224]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:56.153 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:55.863+0000 7f8149b0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:07:56.153 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 bash[38577]: 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 2024-08-01T04:07:56.153 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:55 smithi160 systemd[1]: Started Ceph mgr.y for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:07:56.153 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:56.020+0000 7f8149b0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:07:56.153 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:56.087+0000 7f8149b0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:07:56.295 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:56.091+0000 7f3226cde040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:07:56.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-mon[37383]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:56.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:55 smithi163 ceph-mon[37383]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:56.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:55 smithi003 ceph-mon[32262]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:56.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:55 smithi003 ceph-mon[32262]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:07:56.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:56.341+0000 7f8149b0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:07:56.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:56.403+0000 7f8149b0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:07:56.779 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:56.425+0000 7f3226cde040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:07:56.779 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:56.501+0000 7f3226cde040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:07:56.779 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:56.662+0000 7f3226cde040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:07:57.046 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:56.986+0000 7f3226cde040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:07:57.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:57.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:57.224 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.224 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.224 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.224 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:57.224 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:57.396 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:57.049+0000 7f3226cde040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:07:57.396 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:57.244+0000 7f3226cde040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:07:57.396 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:57.394+0000 7f3226cde040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:07:57.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:57.385+0000 7f8149b0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:07:57.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:57.503+0000 7f8149b0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:07:57.653 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:57.559+0000 7f8149b0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:07:57.795 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:57.461+0000 7f3226cde040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:07:57.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:57.545+0000 7f3226cde040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:07:57.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:57.604+0000 7f3226cde040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:07:58.048 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:58.049 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.050 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:57.775+0000 7f8149b0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:07:58.050 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:57.836+0000 7f8149b0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:07:58.050 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:57.901+0000 7f8149b0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:07:58.050 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:58.047+0000 7f8149b0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:07:58.295 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:58.146+0000 7f3226cde040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:58.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:58.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:58.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:07:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:58.777 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:58.462+0000 7f8149b0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:07:58.777 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:58.775+0000 7f8149b0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:07:58.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:07:58.305+0000 7f3226cde040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:07:59.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/968857622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]: dispatch 2024-08-01T04:07:59.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]: dispatch 2024-08-01T04:07:59.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]': finished 2024-08-01T04:07:59.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: osdmap e5: 1 total, 0 up, 1 in 2024-08-01T04:07:59.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:07:59.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:59.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-01T04:07:59.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:07:59.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: Standby manager daemon z started 2024-08-01T04:07:59.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-01T04:07:59.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:07:59.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:58 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2391684698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:07:59.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:07:59.132+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7fe4f4021950 con 0x7fe504103980 2024-08-01T04:07:59.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:58.841+0000 7f8149b0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:07:59.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:58.990+0000 7f8149b0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/968857622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]: dispatch 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]: dispatch 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]': finished 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: osdmap e5: 1 total, 0 up, 1 in 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: Standby manager daemon z started 2024-08-01T04:07:59.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-01T04:07:59.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:07:59.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:58 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2391684698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:07:59.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/968857622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]: dispatch 2024-08-01T04:07:59.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]: dispatch 2024-08-01T04:07:59.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0fdfd41-e466-4275-8212-709184cca9a6"}]': finished 2024-08-01T04:07:59.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: osdmap e5: 1 total, 0 up, 1 in 2024-08-01T04:07:59.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:07:59.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:07:59.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-01T04:07:59.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:07:59.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: Standby manager daemon z started 2024-08-01T04:07:59.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-01T04:07:59.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/3369803337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:07:59.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:58 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2391684698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:07:59.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.300+0000 7f8149b0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:07:59.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.360+0000 7f8149b0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:07:59.653 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.542+0000 7f8149b0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:07:59.956 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.688+0000 7f8149b0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:07:59.956 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.751+0000 7f8149b0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:07:59.956 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.834+0000 7f8149b0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:07:59.956 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:07:59.891+0000 7f8149b0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:08:00.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:59 smithi163 ceph-mon[37383]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:00.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:59 smithi163 ceph-mon[37383]: mgrmap e14: x(active, since 71s), standbys: z 2024-08-01T04:08:00.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:07:59 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-01T04:08:00.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:59 smithi003 ceph-mon[32262]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:00.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:59 smithi003 ceph-mon[32262]: mgrmap e14: x(active, since 71s), standbys: z 2024-08-01T04:08:00.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:07:59 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-01T04:08:00.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-mon[37224]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:00.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-mon[37224]: mgrmap e14: x(active, since 71s), standbys: z 2024-08-01T04:08:00.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:07:59 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-01T04:08:00.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:08:00.423+0000 7f8149b0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:08:00.903 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:08:00.579+0000 7f8149b0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:08:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:00.975+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mon.1 v2:172.21.15.163:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fe4f404ec10 con 0x7fe504103980 2024-08-01T04:08:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:00 smithi163 ceph-mon[37383]: Standby manager daemon y started 2024-08-01T04:08:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:00 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-01T04:08:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:00 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:08:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:00 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-01T04:08:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:00 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:08:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:00 smithi003 ceph-mon[32262]: Standby manager daemon y started 2024-08-01T04:08:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:00 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-01T04:08:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:00 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:08:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:00 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-01T04:08:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:00 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:08:01.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-mon[37224]: Standby manager daemon y started 2024-08-01T04:08:01.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-01T04:08:01.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:08:01.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-01T04:08:01.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:00 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2795187423' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:08:02.114 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:01 smithi003 ceph-mon[32262]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:02.114 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:01 smithi003 ceph-mon[32262]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-08-01T04:08:02.114 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-01T04:08:02.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:01 smithi163 ceph-mon[37383]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:02.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:01 smithi163 ceph-mon[37383]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-08-01T04:08:02.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-01T04:08:02.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:01 smithi160 ceph-mon[37224]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:02.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:01 smithi160 ceph-mon[37224]: mgrmap e15: x(active, since 72s), standbys: z, y 2024-08-01T04:08:02.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-01T04:08:04.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:03 smithi163 ceph-mon[37383]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:04.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:03 smithi003 ceph-mon[32262]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:04.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:03 smithi160 ceph-mon[37224]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:06.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:05 smithi003 ceph-mon[32262]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:05 smithi163 ceph-mon[37383]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:06.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:05 smithi160 ceph-mon[37224]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:07.234 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:06 smithi003 ceph-mon[32262]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:07.234 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-01T04:08:07.234 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:07.234 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:06 smithi003 ceph-mon[32262]: Deploying daemon osd.0 on smithi003 2024-08-01T04:08:07.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:06 smithi163 ceph-mon[37383]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:07.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-01T04:08:07.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:07.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:06 smithi163 ceph-mon[37383]: Deploying daemon osd.0 on smithi003 2024-08-01T04:08:07.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:06 smithi160 ceph-mon[37224]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:07.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-01T04:08:07.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:07.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:06 smithi160 ceph-mon[37224]: Deploying daemon osd.0 on smithi003 2024-08-01T04:08:09.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:09 smithi003 ceph-mon[32262]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:09.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:09 smithi160 ceph-mon[37224]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:09.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:09 smithi163 ceph-mon[37383]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:11.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:11 smithi003 ceph-mon[32262]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:11.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:11 smithi160 ceph-mon[37224]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:11.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:11 smithi163 ceph-mon[37383]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:13 smithi003 ceph-mon[32262]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:13.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:13 smithi160 ceph-mon[37224]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:13.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:13 smithi163 ceph-mon[37383]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:14.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:14.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:14.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:14.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:15.456 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:15 smithi160 ceph-mon[37224]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:15.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:15 smithi163 ceph-mon[37383]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:15.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:15 smithi003 ceph-mon[32262]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:17 smithi003 ceph-mon[32262]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:17.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:17.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:17.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:17 smithi163 ceph-mon[37383]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:17 smithi160 ceph-mon[37224]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:17.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:17.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:17.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:17.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.206+0000 7fe500c00640 1 -- 172.21.15.3:0/4188087497 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe504062360 con 0x7fe4e8039290 2024-08-01T04:08:18.209 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe4e8039290 msgr2=0x7fe4e803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe4e8039290 0x7fe4e803b750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec030550 comp rx=0 tx=0).stop 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 msgr2=0x7fe504199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 0x7fe504199c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400e9c0 tx=0x7fe4f400ee80 comp rx=0 tx=0).stop 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 shutdown_connections 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe4e8039290 0x7fe4e803b750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe504105540 0x7fe5041a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe504104b80 0x7fe50419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 --2- 172.21.15.3:0/4188087497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe504103980 0x7fe504199c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.209+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 >> 172.21.15.3:0/4188087497 conn(0x7fe5040ff110 msgr2=0x7fe504107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:08:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.210+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 shutdown_connections 2024-08-01T04:08:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:08:18.210+0000 7fe509000640 1 -- 172.21.15.3:0/4188087497 wait complete. 2024-08-01T04:08:18.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:18.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:18.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:18.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:19.011 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.0.service 2024-08-01T04:08:19.017 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_4... 2024-08-01T04:08:19.018 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-01T04:08:19.279 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:08:19.393 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:19 smithi160 ceph-mon[37224]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:19.393 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:19 smithi160 ceph-mon[37224]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-01T04:08:19.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:19 smithi003 ceph-mon[32262]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:19.511 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:19 smithi003 ceph-mon[32262]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-01T04:08:19.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:19 smithi163 ceph-mon[37383]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:19.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:19 smithi163 ceph-mon[37383]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-01T04:08:20.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:20 smithi160 ceph-mon[37224]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-01T04:08:20.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:20 smithi160 ceph-mon[37224]: osdmap e6: 1 total, 0 up, 1 in 2024-08-01T04:08:20.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:20 smithi160 ceph-mon[37224]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:08:20.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:20 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:20.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:20 smithi163 ceph-mon[37383]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-01T04:08:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:20 smithi163 ceph-mon[37383]: osdmap e6: 1 total, 0 up, 1 in 2024-08-01T04:08:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:20 smithi163 ceph-mon[37383]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:08:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:20 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:20.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:20 smithi003 ceph-mon[32262]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-01T04:08:20.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:20 smithi003 ceph-mon[32262]: osdmap e6: 1 total, 0 up, 1 in 2024-08-01T04:08:20.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:20 smithi003 ceph-mon[32262]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:08:20.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:20 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.246 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:08:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0[46099]: 2024-08-01T04:08:20.813+0000 7f7d8d200640 -1 osd.0 0 waiting for initial osdmap 2024-08-01T04:08:21.247 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:08:20 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0[46099]: 2024-08-01T04:08:20.829+0000 7f7d82800640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:08:21.474 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:08:21.496 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:21 smithi160 ceph-mon[37224]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:21 smithi160 ceph-mon[37224]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:08:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:21 smithi160 ceph-mon[37224]: osdmap e7: 1 total, 0 up, 1 in 2024-08-01T04:08:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.497 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:21 smithi160 ceph-mon[37224]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' 2024-08-01T04:08:21.498 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-08-01T04:08:21.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:21 smithi163 ceph-mon[37383]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:21 smithi163 ceph-mon[37383]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:08:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:21 smithi163 ceph-mon[37383]: osdmap e7: 1 total, 0 up, 1 in 2024-08-01T04:08:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:21 smithi163 ceph-mon[37383]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' 2024-08-01T04:08:21.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:21 smithi003 ceph-mon[32262]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:21.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:21 smithi003 ceph-mon[32262]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:08:21.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:21 smithi003 ceph-mon[32262]: osdmap e7: 1 total, 0 up, 1 in 2024-08-01T04:08:21.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:21.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:21 smithi003 ceph-mon[32262]: from='osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457]' entity='osd.0' 2024-08-01T04:08:21.753 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:08:22.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 -- 172.21.15.160:0/1445181683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 msgr2=0x7f102c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 --2- 172.21.15.160:0/1445181683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c100c00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1028009900 tx=0x7f102802a410 comp rx=0 tx=0).stop 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 -- 172.21.15.160:0/1445181683 shutdown_connections 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 --2- 172.21.15.160:0/1445181683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f102c1023c0 0x7f102c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 --2- 172.21.15.160:0/1445181683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 0x7f102c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 --2- 172.21.15.160:0/1445181683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c100c00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 -- 172.21.15.160:0/1445181683 >> 172.21.15.160:0/1445181683 conn(0x7f102c0fbfb0 msgr2=0x7f102c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 -- 172.21.15.160:0/1445181683 shutdown_connections 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 -- 172.21.15.160:0/1445181683 wait complete. 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 Processor -- start 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.164+0000 7f1033d5b640 1 -- start start 2024-08-01T04:08:22.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1033d5b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1033d5b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 0x7f102c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1033d5b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f102c1023c0 0x7f102c103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1033d5b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f102c10c500 con 0x7f102c101a00 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1033d5b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f102c10c380 con 0x7f102c100800 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1033d5b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f102c10c680 con 0x7f102c1023c0 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60882/0 (socket says 172.21.15.160:60882) 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 -- 172.21.15.160:0/4132484656 learned_addr learned my addr 172.21.15.160:0/4132484656 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f10312f6640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 0x7f102c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f10322f8640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f102c1023c0 0x7f102c103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 -- 172.21.15.160:0/4132484656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f102c1023c0 msgr2=0x7f102c103320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:22.169 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f102c1023c0 0x7f102c103320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 -- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 msgr2=0x7f102c102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 0x7f102c102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 -- 172.21.15.160:0/4132484656 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c103be0 con 0x7f102c100800 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f10312f6640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 0x7f102c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f1031af7640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c104790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1028031610 tx=0x7f10280043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.165+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102803f030 con 0x7f102c100800 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.166+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c1a1e40 con 0x7f102c100800 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.166+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1028003ce0 con 0x7f102c100800 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.166+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028035070 con 0x7f102c100800 2024-08-01T04:08:22.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.166+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c1a2260 con 0x7f102c100800 2024-08-01T04:08:22.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.167+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c1a2490 con 0x7f102c100800 2024-08-01T04:08:22.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.168+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f1028042030 con 0x7f102c100800 2024-08-01T04:08:22.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.168+0000 7f101effd640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f10180a2af0 0x7f10180a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:22.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.168+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f10280edee0 con 0x7f102c100800 2024-08-01T04:08:22.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.169+0000 7f10312f6640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f10180a2af0 0x7f10180a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:22.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.169+0000 7f10312f6640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f10180a2af0 0x7f10180a4fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1020006fd0 tx=0x7f1020008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:08:22.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.172+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f10280bc8b0 con 0x7f102c100800 2024-08-01T04:08:22.353 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:22 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:08:22.353 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:22 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:08:22.353 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:22 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:22.353 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:22 smithi160 ceph-mon[37224]: osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] boot 2024-08-01T04:08:22.353 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:22 smithi160 ceph-mon[37224]: osdmap e8: 1 total, 1 up, 1 in 2024-08-01T04:08:22.353 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:22 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:22.354 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:22.351+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f102c1a2720 con 0x7f10180a2af0 2024-08-01T04:08:22.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:22 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:08:22.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:22 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:08:22.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:22 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:22.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:22 smithi003 ceph-mon[32262]: osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] boot 2024-08-01T04:08:22.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:22 smithi003 ceph-mon[32262]: osdmap e8: 1 total, 1 up, 1 in 2024-08-01T04:08:22.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:22 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:22 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:08:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:22 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:08:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:22 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:22 smithi163 ceph-mon[37383]: osd.0 [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] boot 2024-08-01T04:08:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:22 smithi163 ceph-mon[37383]: osdmap e8: 1 total, 1 up, 1 in 2024-08-01T04:08:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:22 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:08:23.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:23 smithi160 ceph-mon[37224]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:23.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:23 smithi160 ceph-mon[37224]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:08:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:08:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:08:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:23.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:23 smithi163 ceph-mon[37383]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:23 smithi163 ceph-mon[37383]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:08:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:08:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:08:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:23.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:23 smithi003 ceph-mon[32262]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-01T04:08:23.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:23 smithi003 ceph-mon[32262]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:08:23.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:08:23.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:08:23.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: osdmap e9: 1 total, 1 up, 1 in 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi003 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi003 to 17094M 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: osdmap e9: 1 total, 1 up, 1 in 2024-08-01T04:08:24.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi003 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi003 to 17094M 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: osdmap e9: 1 total, 1 up, 1 in 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi003 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi003 to 17094M 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:24.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:25.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:25.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/927790028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]: dispatch 2024-08-01T04:08:25.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]: dispatch 2024-08-01T04:08:25.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]': finished 2024-08-01T04:08:25.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: osdmap e10: 2 total, 1 up, 2 in 2024-08-01T04:08:25.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:25.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/1890209895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/927790028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]: dispatch 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]: dispatch 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]': finished 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: osdmap e10: 2 total, 1 up, 2 in 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/1890209895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:08:25.799 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:25.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/927790028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]: dispatch 2024-08-01T04:08:25.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]: dispatch 2024-08-01T04:08:25.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9b159fc-e987-4b22-b63e-910d2fc377e3"}]': finished 2024-08-01T04:08:25.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: osdmap e10: 2 total, 1 up, 2 in 2024-08-01T04:08:25.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:25.800 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/1890209895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:08:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:27 smithi160 ceph-mon[37224]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:27 smithi163 ceph-mon[37383]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:27 smithi003 ceph-mon[32262]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:29.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:29 smithi160 ceph-mon[37224]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:29.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:29 smithi163 ceph-mon[37383]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:29.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:29 smithi003 ceph-mon[32262]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:31.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:31 smithi160 ceph-mon[37224]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:31.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-01T04:08:31.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:31.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:31 smithi163 ceph-mon[37383]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:31.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-01T04:08:31.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:31.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:31 smithi003 ceph-mon[32262]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:31.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-01T04:08:31.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:32.537 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:32 smithi160 ceph-mon[37224]: Deploying daemon osd.1 on smithi160 2024-08-01T04:08:32.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:32 smithi163 ceph-mon[37383]: Deploying daemon osd.1 on smithi160 2024-08-01T04:08:32.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:32 smithi003 ceph-mon[32262]: Deploying daemon osd.1 on smithi160 2024-08-01T04:08:33.624 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:33 smithi160 ceph-mon[37224]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:33.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:33 smithi003 ceph-mon[32262]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:33.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:33 smithi163 ceph-mon[37383]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:35.608 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:35 smithi160 ceph-mon[37224]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:35.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:35 smithi163 ceph-mon[37383]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:35.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:35 smithi003 ceph-mon[32262]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:36.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:36.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:36.517 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:36.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:36.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:36.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:36.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:36.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:36.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:37 smithi160 ceph-mon[37224]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:37 smithi163 ceph-mon[37383]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:37 smithi003 ceph-mon[32262]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:39.247 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-08-01T04:08:39.248 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.245+0000 7f101effd640 1 -- 172.21.15.160:0/4132484656 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f102c1a2720 con 0x7f10180a2af0 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f10180a2af0 msgr2=0x7f10180a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f10180a2af0 0x7f10180a4fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1020006fd0 tx=0x7f1020008040 comp rx=0 tx=0).stop 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 msgr2=0x7f102c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c104790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1028031610 tx=0x7f10280043b0 comp rx=0 tx=0).stop 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 shutdown_connections 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f10180a2af0 0x7f10180a4fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f102c1023c0 0x7f102c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f102c101a00 0x7f102c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 --2- 172.21.15.160:0/4132484656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102c100800 0x7f102c104790 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:39.249 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 >> 172.21.15.160:0/4132484656 conn(0x7f102c0fbfb0 msgr2=0x7f102c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:08:39.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 shutdown_connections 2024-08-01T04:08:39.250 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:08:39.247+0000 7f1033d5b640 1 -- 172.21.15.160:0/4132484656 wait complete. 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.625 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:39.626 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.626 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:39.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:39.856 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.1.service 2024-08-01T04:08:39.859 INFO:tasks.cephadm:Deploying osd.2 on smithi163 with /dev/vg_nvme/lv_4... 2024-08-01T04:08:39.859 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-01T04:08:40.117 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:08:40.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:40 smithi163 ceph-mon[37383]: from='osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-01T04:08:40.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:40 smithi163 ceph-mon[37383]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-01T04:08:40.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:40 smithi160 ceph-mon[37224]: from='osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-01T04:08:40.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:40 smithi160 ceph-mon[37224]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-01T04:08:40.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:40 smithi003 ceph-mon[32262]: from='osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-01T04:08:40.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:40 smithi003 ceph-mon[32262]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-01T04:08:41.422 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:41 smithi163 ceph-mon[37383]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:41.422 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:41 smithi163 ceph-mon[37383]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-01T04:08:41.422 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:41 smithi163 ceph-mon[37383]: from='osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:08:41.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-mon[37224]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:41.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-mon[37224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-01T04:08:41.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-mon[37224]: from='osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:08:41.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-mon[37224]: osdmap e11: 2 total, 1 up, 2 in 2024-08-01T04:08:41.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:41.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-mon[37224]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:08:41.653 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1[41657]: 2024-08-01T04:08:41.413+0000 7f545f277640 -1 osd.1 0 waiting for initial osdmap 2024-08-01T04:08:41.653 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:08:41 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1[41657]: 2024-08-01T04:08:41.416+0000 7f545b09f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:08:41.759 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:41 smithi163 ceph-mon[37383]: osdmap e11: 2 total, 1 up, 2 in 2024-08-01T04:08:41.760 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:41.760 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:41 smithi163 ceph-mon[37383]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:08:41.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:41 smithi003 ceph-mon[32262]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:41.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:41 smithi003 ceph-mon[32262]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-01T04:08:41.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:41 smithi003 ceph-mon[32262]: from='osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:08:41.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:41 smithi003 ceph-mon[32262]: osdmap e11: 2 total, 1 up, 2 in 2024-08-01T04:08:41.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:41.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:41 smithi003 ceph-mon[32262]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:08:42.430 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:08:42.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_4 2024-08-01T04:08:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:42 smithi160 ceph-mon[37224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:08:42.526 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:42 smithi160 ceph-mon[37224]: osdmap e12: 2 total, 1 up, 2 in 2024-08-01T04:08:42.526 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:42 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:42.526 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:42 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:42.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:42 smithi163 ceph-mon[37383]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:08:42.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:42 smithi163 ceph-mon[37383]: osdmap e12: 2 total, 1 up, 2 in 2024-08-01T04:08:42.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:42 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:42.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:42 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:42.726 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:08:42.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:42 smithi003 ceph-mon[32262]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:08:42.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:42 smithi003 ceph-mon[32262]: osdmap e12: 2 total, 1 up, 2 in 2024-08-01T04:08:42.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:42 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:42.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:42 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:43.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 -- 172.21.15.163:0/3130668786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 msgr2=0x7f1660108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:43.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 --2- 172.21.15.163:0/3130668786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f1660108c50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1658009ca0 tx=0x7f165802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:08:43.192 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 -- 172.21.15.163:0/3130668786 shutdown_connections 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 --2- 172.21.15.163:0/3130668786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f1660108c50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 --2- 172.21.15.163:0/3130668786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1660101a00 0x7f1660101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 --2- 172.21.15.163:0/3130668786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1660100800 0x7f1660100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 -- 172.21.15.163:0/3130668786 >> 172.21.15.163:0/3130668786 conn(0x7f16600fbfb0 msgr2=0x7f16600fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 -- 172.21.15.163:0/3130668786 shutdown_connections 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.188+0000 7f166693e640 1 -- 172.21.15.163:0/3130668786 wait complete. 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 Processor -- start 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 -- start start 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1660100800 0x7f1660197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1660101a00 0x7f16601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:43.193 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f166019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f166010d610 con 0x7f1660100800 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f166010d490 con 0x7f1660101a00 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f166693e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f166010d790 con 0x7f16601023c0 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f166019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f166019f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51088/0 (socket says 172.21.15.163:51088) 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 -- 172.21.15.163:0/3670140269 learned_addr learned my addr 172.21.15.163:0/3670140269 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f165f7fe640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1660101a00 0x7f16601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:43.194 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 -- 172.21.15.163:0/3670140269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1660101a00 msgr2=0x7f16601981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1660101a00 0x7f16601981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 -- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1660100800 msgr2=0x7f1660197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1660100800 0x7f1660197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.189+0000 7f1664edb640 1 -- 172.21.15.163:0/3670140269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166019f880 con 0x7f16601023c0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.190+0000 7f1664edb640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f166019f210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1658005e30 tx=0x7f1658031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.190+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658003c30 con 0x7f16601023c0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.190+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166019fb10 con 0x7f16601023c0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.190+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1658003dd0 con 0x7f16601023c0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.190+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165802fde0 con 0x7f16601023c0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.190+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601a0120 con 0x7f16601023c0 2024-08-01T04:08:43.195 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.191+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1660107ba0 con 0x7f16601023c0 2024-08-01T04:08:43.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.192+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f1658004050 con 0x7f16601023c0 2024-08-01T04:08:43.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.193+0000 7f165d7fa640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16480a2af0 0x7f16480a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:08:43.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.193+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f16580eed50 con 0x7f16601023c0 2024-08-01T04:08:43.196 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.193+0000 7f165ffff640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16480a2af0 0x7f16480a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:08:43.197 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.194+0000 7f165ffff640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16480a2af0 0x7f16480a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1660198b40 tx=0x7f1650006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:08:43.197 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.196+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f16580bd500 con 0x7f16601023c0 2024-08-01T04:08:43.381 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:08:43.379+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1660062360 con 0x7f16480a2af0 2024-08-01T04:08:43.718 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:08:43.718 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] boot 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: osdmap e13: 2 total, 2 up, 2 in 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi160 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi160 to 17088M 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:43.719 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.720 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:08:43.720 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:08:43.720 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:43.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:08:43.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] boot 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: osdmap e13: 2 total, 2 up, 2 in 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi160 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi160 to 17088M 2024-08-01T04:08:43.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:43.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:43.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:08:43.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:08:43.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:08:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:08:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-01T04:08:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: osd.1 [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] boot 2024-08-01T04:08:43.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: osdmap e13: 2 total, 2 up, 2 in 2024-08-01T04:08:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:08:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi160 2024-08-01T04:08:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:08:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi160 to 17088M 2024-08-01T04:08:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:08:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:43.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:08:43.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:08:43.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:44 smithi160 ceph-mon[37224]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:08:44.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:44 smithi160 ceph-mon[37224]: osdmap e14: 2 total, 2 up, 2 in 2024-08-01T04:08:44.914 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:44 smithi163 ceph-mon[37383]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:08:44.914 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:44 smithi163 ceph-mon[37383]: osdmap e14: 2 total, 2 up, 2 in 2024-08-01T04:08:45.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:44 smithi003 ceph-mon[32262]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:08:45.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:44 smithi003 ceph-mon[32262]: osdmap e14: 2 total, 2 up, 2 in 2024-08-01T04:08:45.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:45 smithi003 ceph-mon[32262]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:45.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:45 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/1832077983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]: dispatch 2024-08-01T04:08:45.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:45 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]: dispatch 2024-08-01T04:08:45.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:45 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]': finished 2024-08-01T04:08:45.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:45 smithi003 ceph-mon[32262]: osdmap e15: 3 total, 2 up, 3 in 2024-08-01T04:08:45.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:08:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:45 smithi160 ceph-mon[37224]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:45 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/1832077983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]: dispatch 2024-08-01T04:08:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:45 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]: dispatch 2024-08-01T04:08:45.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:45 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]': finished 2024-08-01T04:08:45.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:45 smithi160 ceph-mon[37224]: osdmap e15: 3 total, 2 up, 3 in 2024-08-01T04:08:45.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:08:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:45 smithi163 ceph-mon[37383]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:45 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/1832077983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]: dispatch 2024-08-01T04:08:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:45 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]: dispatch 2024-08-01T04:08:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:45 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90f0ad0d-8599-4670-868b-e0b0a593aa0d"}]': finished 2024-08-01T04:08:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:45 smithi163 ceph-mon[37383]: osdmap e15: 3 total, 2 up, 3 in 2024-08-01T04:08:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:08:46.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:46 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/1119832216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:08:47.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:46 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/1119832216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:08:47.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:46 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/1119832216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:08:47.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:47 smithi160 ceph-mon[37224]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:48.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:47 smithi163 ceph-mon[37383]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:47 smithi003 ceph-mon[32262]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:49 smithi160 ceph-mon[37224]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:50.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:49 smithi163 ceph-mon[37383]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:49 smithi003 ceph-mon[32262]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:51.880 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:51 smithi163 ceph-mon[37383]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:51 smithi160 ceph-mon[37224]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:52.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:51 smithi003 ceph-mon[32262]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:52.871 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-01T04:08:52.871 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:52.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-01T04:08:52.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:53.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-01T04:08:53.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:08:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:53 smithi160 ceph-mon[37224]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:53 smithi160 ceph-mon[37224]: Deploying daemon osd.2 on smithi163 2024-08-01T04:08:53.931 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:53 smithi163 ceph-mon[37383]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:53.931 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:53 smithi163 ceph-mon[37383]: Deploying daemon osd.2 on smithi163 2024-08-01T04:08:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:53 smithi003 ceph-mon[32262]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:53 smithi003 ceph-mon[32262]: Deploying daemon osd.2 on smithi163 2024-08-01T04:08:55.885 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:55 smithi163 ceph-mon[37383]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:56.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:55 smithi003 ceph-mon[32262]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:55 smithi160 ceph-mon[37224]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:57.964 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:57 smithi163 ceph-mon[37383]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:57.964 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:57.964 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:57.964 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:57 smithi003 ceph-mon[32262]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:58.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:57 smithi160 ceph-mon[37224]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:08:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:08:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:08:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:00.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:08:59 smithi163 ceph-mon[37383]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:00.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:08:59 smithi003 ceph-mon[32262]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:08:59 smithi160 ceph-mon[37224]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:00.518 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.516+0000 7f165d7fa640 1 -- 172.21.15.163:0/3670140269 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1660062360 con 0x7f16480a2af0 2024-08-01T04:09:00.518 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 2 on host 'smithi163' 2024-08-01T04:09:00.520 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16480a2af0 msgr2=0x7f16480a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:00.520 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16480a2af0 0x7f16480a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1660198b40 tx=0x7f1650006c90 comp rx=0 tx=0).stop 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 msgr2=0x7f166019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f166019f210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1658005e30 tx=0x7f1658031ee0 comp rx=0 tx=0).stop 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 shutdown_connections 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16480a2af0 0x7f16480a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16601023c0 0x7f166019f210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1660101a00 0x7f16601981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 --2- 172.21.15.163:0/3670140269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1660100800 0x7f1660197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 >> 172.21.15.163:0/3670140269 conn(0x7f16600fbfb0 msgr2=0x7f1660104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 shutdown_connections 2024-08-01T04:09:00.521 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:00.518+0000 7f166693e640 1 -- 172.21.15.163:0/3670140269 wait complete. 2024-08-01T04:09:01.116 DEBUG:teuthology.orchestra.run.smithi163:osd.2> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.2.service 2024-08-01T04:09:01.119 INFO:tasks.cephadm:Deploying osd.3 on smithi003 with /dev/vg_nvme/lv_3... 2024-08-01T04:09:01.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-01T04:09:01.506 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2[42253]: 2024-08-01T04:09:01.173+0000 7ff861e67640 -1 osd.2 0 log_to_monitors true 2024-08-01T04:09:01.591 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: from='osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.786 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: from='osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-01T04:09:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-01T04:09:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: from='osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:01.799 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:02.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:02 smithi003 ceph-mon[32262]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-01T04:09:02.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:02 smithi003 ceph-mon[32262]: osdmap e16: 3 total, 2 up, 3 in 2024-08-01T04:09:02.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:02 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:02.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:02 smithi003 ceph-mon[32262]: from='osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:09:02.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:02 smithi003 ceph-mon[32262]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:09:02.868 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2[42253]: 2024-08-01T04:09:02.680+0000 7ff85ddea640 -1 osd.2 0 waiting for initial osdmap 2024-08-01T04:09:02.869 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2[42253]: 2024-08-01T04:09:02.687+0000 7ff859c12640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:09:02.869 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-mon[37383]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-01T04:09:02.869 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-mon[37383]: osdmap e16: 3 total, 2 up, 3 in 2024-08-01T04:09:02.869 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:02.869 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-mon[37383]: from='osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:09:02.870 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:02 smithi163 ceph-mon[37383]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:09:03.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:02 smithi160 ceph-mon[37224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-01T04:09:03.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:02 smithi160 ceph-mon[37224]: osdmap e16: 3 total, 2 up, 3 in 2024-08-01T04:09:03.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:02 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:03.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:02 smithi160 ceph-mon[37224]: from='osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:09:03.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:02 smithi160 ceph-mon[37224]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:09:03.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:03 smithi163 ceph-mon[37383]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:03.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:03 smithi163 ceph-mon[37383]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:09:03.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:03 smithi163 ceph-mon[37383]: osdmap e17: 3 total, 2 up, 3 in 2024-08-01T04:09:03.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:03 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:03.958 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:03 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:04.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:03 smithi003 ceph-mon[32262]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:04.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:03 smithi003 ceph-mon[32262]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:09:04.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:03 smithi003 ceph-mon[32262]: osdmap e17: 3 total, 2 up, 3 in 2024-08-01T04:09:04.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:03 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:04.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:03 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:04.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:03 smithi160 ceph-mon[37224]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:04.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:03 smithi160 ceph-mon[37224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:09:04.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:03 smithi160 ceph-mon[37224]: osdmap e17: 3 total, 2 up, 3 in 2024-08-01T04:09:04.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:03 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:04.054 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:03 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:04 smithi163 ceph-mon[37383]: osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] boot 2024-08-01T04:09:05.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:04 smithi163 ceph-mon[37383]: osdmap e18: 3 total, 3 up, 3 in 2024-08-01T04:09:05.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:04 smithi003 ceph-mon[32262]: osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] boot 2024-08-01T04:09:05.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:04 smithi003 ceph-mon[32262]: osdmap e18: 3 total, 3 up, 3 in 2024-08-01T04:09:05.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:04 smithi160 ceph-mon[37224]: osd.2 [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] boot 2024-08-01T04:09:05.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:04 smithi160 ceph-mon[37224]: osdmap e18: 3 total, 3 up, 3 in 2024-08-01T04:09:05.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:09:05.260 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:09:05.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_3 2024-08-01T04:09:05.750 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi163 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: osdmap e19: 3 total, 3 up, 3 in 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi163 to 17088M 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi163 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: osdmap e19: 3 total, 3 up, 3 in 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi163 to 17088M 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:09:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi163 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: osdmap e19: 3 total, 3 up, 3 in 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi163 to 17088M 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:06.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:06.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:06.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:06.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:06.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2188796836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 msgr2=0x7f8c181098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2188796836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 0x7f8c181098c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009ca0 tx=0x7f8c1002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2188796836 shutdown_connections 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2188796836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 0x7f8c181098c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2188796836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 0x7f8c18106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2188796836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 0x7f8c18104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2188796836 >> 172.21.15.3:0/2188796836 conn(0x7f8c180fbe10 msgr2=0x7f8c180fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.336+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2188796836 shutdown_connections 2024-08-01T04:09:06.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.337+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2188796836 wait complete. 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.337+0000 7f8c1ca00640 1 Processor -- start 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 -- start start 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 0x7f8c18199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 0x7f8c1819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 0x7f8c181a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1810d410 con 0x7f8c18102210 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1810d290 con 0x7f8c18104b40 2024-08-01T04:09:06.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c1ca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1810d590 con 0x7f8c18107470 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c0fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 0x7f8c1819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c17400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 0x7f8c18199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c0fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 0x7f8c1819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:48096/0 (socket says 172.21.15.3:48096) 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c0fe00640 1 -- 172.21.15.3:0/2568416324 learned_addr learned my addr 172.21.15.3:0/2568416324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.338+0000 7f8c17e00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 0x7f8c181a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.339+0000 7f8c17400640 1 -- 172.21.15.3:0/2568416324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 msgr2=0x7f8c181a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.339+0000 7f8c17400640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 0x7f8c181a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.339+0000 7f8c17400640 1 -- 172.21.15.3:0/2568416324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 msgr2=0x7f8c1819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.339+0000 7f8c17400640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 0x7f8c1819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.339+0000 7f8c17400640 1 -- 172.21.15.3:0/2568416324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c181a1860 con 0x7f8c18102210 2024-08-01T04:09:06.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.339+0000 7f8c17400640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 0x7f8c18199c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800e9f0 tx=0x7f8c0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:09:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.340+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800cd80 con 0x7f8c18102210 2024-08-01T04:09:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.340+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c080035f0 con 0x7f8c18102210 2024-08-01T04:09:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.340+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08010690 con 0x7f8c18102210 2024-08-01T04:09:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.340+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c181a1b50 con 0x7f8c18102210 2024-08-01T04:09:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.340+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c181a20e0 con 0x7f8c18102210 2024-08-01T04:09:06.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.341+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c18108f60 con 0x7f8c18102210 2024-08-01T04:09:06.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.344+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f8c08003b30 con 0x7f8c18102210 2024-08-01T04:09:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.346+0000 7f8c14c00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f8bfc0a2b70 0x7f8bfc0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:06.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.347+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f8c080cdee0 con 0x7f8c18102210 2024-08-01T04:09:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.347+0000 7f8c0fe00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f8bfc0a2b70 0x7f8bfc0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.348+0000 7f8c0fe00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f8bfc0a2b70 0x7f8bfc0a5030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c000058c0 tx=0x7f8c00005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:09:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.350+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8c0809c2f0 con 0x7f8c18102210 2024-08-01T04:09:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.552+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8c18062360 con 0x7f8bfc0a2b70 2024-08-01T04:09:06.751 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:06 smithi003 sudo[50418]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-01T04:09:06.752 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:06 smithi003 sudo[50418]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:06.752 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:06 smithi003 sudo[50418]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:06.752 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:06 smithi003 sudo[50418]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:06.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:06.761+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8c08098260 con 0x7f8c18102210 2024-08-01T04:09:07.001 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:06 smithi003 sudo[50499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-01T04:09:07.001 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:07 smithi003 sudo[50499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:07.001 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:07 smithi003 sudo[50499]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:07.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-01T04:09:07.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:06 smithi003 ceph-mon[32262]: osdmap e20: 3 total, 3 up, 3 in 2024-08-01T04:09:07.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:07.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:07.002 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-01T04:09:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:06 smithi163 ceph-mon[37383]: osdmap e20: 3 total, 3 up, 3 in 2024-08-01T04:09:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:07.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-01T04:09:07.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:06 smithi160 ceph-mon[37224]: osdmap e20: 3 total, 3 up, 3 in 2024-08-01T04:09:07.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:07.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:07.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:07.252 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:09:07 smithi003 sudo[50499]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:07.541 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45507]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-01T04:09:07.541 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45507]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:07.541 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45507]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:07.541 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45507]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:07 smithi003 ceph-mon[32262]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:07 smithi003 ceph-mon[32262]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:07 smithi003 ceph-mon[32262]: osdmap e21: 3 total, 3 up, 3 in 2024-08-01T04:09:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:07 smithi003 ceph-mon[32262]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-01T04:09:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:07.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45510]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-01T04:09:07.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45510]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:07.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45510]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:07.903 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:09:07 smithi160 sudo[45510]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:07.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:07 smithi160 ceph-mon[37224]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:07.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:07 smithi160 ceph-mon[37224]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:07.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:07 smithi160 ceph-mon[37224]: osdmap e21: 3 total, 3 up, 3 in 2024-08-01T04:09:07.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:07 smithi160 ceph-mon[37224]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-01T04:09:07.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:07.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:07 smithi163 ceph-mon[37383]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:07 smithi163 ceph-mon[37383]: from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:07 smithi163 ceph-mon[37383]: osdmap e21: 3 total, 3 up, 3 in 2024-08-01T04:09:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:07 smithi163 ceph-mon[37383]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-01T04:09:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:08.046 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:07 smithi163 sudo[46300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-01T04:09:08.047 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:07 smithi163 sudo[46300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:08.047 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:07 smithi163 sudo[46300]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:08.047 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:07 smithi163 sudo[46300]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:08.545 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:08 smithi163 sudo[46303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-01T04:09:08.546 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:08 smithi163 sudo[46303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:08.546 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:08 smithi163 sudo[46303]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:08.546 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:09:08 smithi163 sudo[46303]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:08.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:08 smithi003 sudo[50584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-01T04:09:08.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:08 smithi003 sudo[50584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:08.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:08 smithi003 sudo[50584]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:08.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:08 smithi003 sudo[50584]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:09.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:09 smithi003 sudo[50594]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-01T04:09:09.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:09 smithi003 sudo[50594]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:09.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:09 smithi003 sudo[50594]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:09.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:09 smithi003 sudo[50594]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:09.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:09.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:09 smithi003 ceph-mon[32262]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:09.562 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 ceph-mon[37383]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-01T04:09:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46306]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:09.563 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46306]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:09 smithi160 ceph-mon[37224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46309]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-01T04:09:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46309]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46309]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:09 smithi163 sudo[46309]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:10.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45513]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-01T04:09:10.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45513]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:10.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45513]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:10.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45513]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45516]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45516]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45516]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 sudo[45516]: pam_unix(sudo:session): session closed for user root 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2418612447' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]: dispatch 2024-08-01T04:09:10.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:10 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]: dispatch 2024-08-01T04:09:10.683 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:10.683 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:10.683 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2418612447' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]: dispatch 2024-08-01T04:09:10.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:10 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]: dispatch 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2418612447' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]: dispatch 2024-08-01T04:09:10.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:10 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]: dispatch 2024-08-01T04:09:11.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]': finished 2024-08-01T04:09:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: osdmap e22: 4 total, 3 up, 4 in 2024-08-01T04:09:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:11 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1352682563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:09:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]': finished 2024-08-01T04:09:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: osdmap e22: 4 total, 3 up, 4 in 2024-08-01T04:09:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:11 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1352682563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97d21a34-b1bd-4f38-ac45-28f4f9360b54"}]': finished 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: osdmap e22: 4 total, 3 up, 4 in 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:11 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1352682563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:09:13.598 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:13 smithi163 ceph-mon[37383]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:13.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:13 smithi160 ceph-mon[37224]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:13 smithi003 ceph-mon[32262]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:15.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:15 smithi003 ceph-mon[32262]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-08-01T04:09:15.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:15 smithi160 ceph-mon[37224]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-08-01T04:09:15.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:15 smithi163 ceph-mon[37383]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-08-01T04:09:17.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:17 smithi003 ceph-mon[32262]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-08-01T04:09:17.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:17 smithi160 ceph-mon[37224]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-08-01T04:09:17.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:17 smithi163 ceph-mon[37383]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-08-01T04:09:19.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:19 smithi160 ceph-mon[37224]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-08-01T04:09:19.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-01T04:09:19.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:19.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:19 smithi163 ceph-mon[37383]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-08-01T04:09:19.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-01T04:09:19.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:19 smithi003 ceph-mon[32262]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-08-01T04:09:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-01T04:09:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:20 smithi003 ceph-mon[32262]: Deploying daemon osd.3 on smithi003 2024-08-01T04:09:20.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:20 smithi160 ceph-mon[37224]: Deploying daemon osd.3 on smithi003 2024-08-01T04:09:20.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:20 smithi163 ceph-mon[37383]: Deploying daemon osd.3 on smithi003 2024-08-01T04:09:21.648 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:21 smithi003 ceph-mon[32262]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-01T04:09:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:21 smithi160 ceph-mon[37224]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-01T04:09:21.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:21 smithi163 ceph-mon[37383]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-01T04:09:23.506 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:23 smithi003 ceph-mon[32262]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 868 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-01T04:09:23.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:23 smithi160 ceph-mon[37224]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 868 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-01T04:09:23.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:23 smithi163 ceph-mon[37383]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 868 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-01T04:09:25.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:25 smithi003 ceph-mon[32262]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-01T04:09:25.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:25 smithi160 ceph-mon[37224]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-01T04:09:25.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:25 smithi163 ceph-mon[37383]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-01T04:09:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:27 smithi160 ceph-mon[37224]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:27.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:27.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:27.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:27 smithi163 ceph-mon[37383]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:27.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:27.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:27.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:27 smithi003 ceph-mon[32262]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:29.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:29 smithi003 ceph-mon[32262]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:29 smithi160 ceph-mon[37224]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:29 smithi163 ceph-mon[37383]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:31.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.724+0000 7f8c14c00640 1 -- 172.21.15.3:0/2568416324 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8c18062360 con 0x7f8bfc0a2b70 2024-08-01T04:09:31.727 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 3 on host 'smithi003' 2024-08-01T04:09:31.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f8bfc0a2b70 msgr2=0x7f8bfc0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:31.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f8bfc0a2b70 0x7f8bfc0a5030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c000058c0 tx=0x7f8c00005f90 comp rx=0 tx=0).stop 2024-08-01T04:09:31.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 msgr2=0x7f8c18199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:31.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 0x7f8c18199c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800e9f0 tx=0x7f8c0800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:09:31.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 shutdown_connections 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f8bfc0a2b70 0x7f8bfc0a5030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c18107470 0x7f8c181a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c18104b40 0x7f8c1819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 --2- 172.21.15.3:0/2568416324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c18102210 0x7f8c18199c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 >> 172.21.15.3:0/2568416324 conn(0x7f8c180fbe10 msgr2=0x7f8c18101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 shutdown_connections 2024-08-01T04:09:31.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:09:31.728+0000 7f8c1ca00640 1 -- 172.21.15.3:0/2568416324 wait complete. 2024-08-01T04:09:31.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:31.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:31.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:31.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:31.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:31.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:31.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:31 smithi160 ceph-mon[37224]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:31 smithi163 ceph-mon[37383]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:31 smithi003 ceph-mon[32262]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-01T04:09:32.504 DEBUG:teuthology.orchestra.run.smithi003:osd.3> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.3.service 2024-08-01T04:09:32.509 INFO:tasks.cephadm:Deploying osd.4 on smithi160 with /dev/vg_nvme/lv_3... 2024-08-01T04:09:32.510 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-01T04:09:32.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-01T04:09:32.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: osdmap e23: 4 total, 3 up, 4 in 2024-08-01T04:09:32.701 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:09:32.701 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:32.701 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:32.701 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.701 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.701 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3[52658]: 2024-08-01T04:09:32.679+0000 7fdfc5c00640 -1 osd.3 0 waiting for initial osdmap 2024-08-01T04:09:32.701 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:09:32 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3[52658]: 2024-08-01T04:09:32.697+0000 7fdfc0c00640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:09:32.772 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:09:32.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-01T04:09:32.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: osdmap e23: 4 total, 3 up, 4 in 2024-08-01T04:09:32.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:09:32.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:32.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:32.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:32.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: osdmap e23: 4 total, 3 up, 4 in 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:33.751 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:33 smithi003 ceph-mon[32262]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:33.752 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:33 smithi003 ceph-mon[32262]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:09:33.752 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:33 smithi003 ceph-mon[32262]: osdmap e24: 4 total, 3 up, 4 in 2024-08-01T04:09:33.752 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:33.752 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:33.925 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:33 smithi160 ceph-mon[37224]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:33.925 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:33 smithi160 ceph-mon[37224]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:09:33.926 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:33 smithi160 ceph-mon[37224]: osdmap e24: 4 total, 3 up, 4 in 2024-08-01T04:09:33.926 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:33 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:33.926 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:33 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:33 smithi163 ceph-mon[37383]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-01T04:09:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:33 smithi163 ceph-mon[37383]: from='osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:09:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:33 smithi163 ceph-mon[37383]: osdmap e24: 4 total, 3 up, 4 in 2024-08-01T04:09:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:34.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:34 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:09:34.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:34 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:09:34.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:34 smithi003 ceph-mon[32262]: osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] boot 2024-08-01T04:09:34.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:34 smithi003 ceph-mon[32262]: osdmap e25: 4 total, 4 up, 4 in 2024-08-01T04:09:34.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:34 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:34.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:34 smithi003 ceph-mon[32262]: osdmap e26: 4 total, 4 up, 4 in 2024-08-01T04:09:34.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:34 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:09:34.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:34 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:09:34.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:34 smithi160 ceph-mon[37224]: osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] boot 2024-08-01T04:09:34.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:34 smithi160 ceph-mon[37224]: osdmap e25: 4 total, 4 up, 4 in 2024-08-01T04:09:34.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:34 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:34.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:34 smithi160 ceph-mon[37224]: osdmap e26: 4 total, 4 up, 4 in 2024-08-01T04:09:35.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:34 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:09:35.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:34 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:09:35.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:34 smithi163 ceph-mon[37383]: osd.3 [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] boot 2024-08-01T04:09:35.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:34 smithi163 ceph-mon[37383]: osdmap e25: 4 total, 4 up, 4 in 2024-08-01T04:09:35.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:34 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:09:35.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:34 smithi163 ceph-mon[37383]: osdmap e26: 4 total, 4 up, 4 in 2024-08-01T04:09:35.154 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:09:35.180 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-08-01T04:09:35.437 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:09:35.788 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:35 smithi160 ceph-mon[37224]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:35.788 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:35 smithi160 ceph-mon[37224]: osdmap e27: 4 total, 4 up, 4 in 2024-08-01T04:09:35.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 -- 172.21.15.160:0/3422449122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 msgr2=0x7efe24101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:35.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/3422449122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe24101e80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efe14009900 tx=0x7efe1402a410 comp rx=0 tx=0).stop 2024-08-01T04:09:35.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 -- 172.21.15.160:0/3422449122 shutdown_connections 2024-08-01T04:09:35.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/3422449122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 0x7efe24108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:35.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/3422449122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe24101e80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:35.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/3422449122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe24100800 0x7efe24100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.871+0000 7efe2b9c6640 1 -- 172.21.15.160:0/3422449122 >> 172.21.15.160:0/3422449122 conn(0x7efe240fbfb0 msgr2=0x7efe240fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 -- 172.21.15.160:0/3422449122 shutdown_connections 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 -- 172.21.15.160:0/3422449122 wait complete. 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 Processor -- start 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 -- start start 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe24100800 0x7efe24197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe241981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 0x7efe2419f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:35.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe2410c480 con 0x7efe241023c0 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efe2410c300 con 0x7efe24101a00 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe2b9c6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efe2410c600 con 0x7efe24100800 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe28f61640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe241981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.872+0000 7efe28f61640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe241981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40660/0 (socket says 172.21.15.160:40660) 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 -- 172.21.15.160:0/2929180103 learned_addr learned my addr 172.21.15.160:0/2929180103 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe29762640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe24100800 0x7efe24197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe29f63640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 0x7efe2419f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 -- 172.21.15.160:0/2929180103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe24100800 msgr2=0x7efe24197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe24100800 0x7efe24197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 -- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 msgr2=0x7efe2419f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:35.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 0x7efe2419f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 -- 172.21.15.160:0/2929180103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2419f880 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe29f63640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 0x7efe2419f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe28f61640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe241981a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe14009230 tx=0x7efe14031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1403f030 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2419fb10 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe14003ad0 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14035070 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.873+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2419fff0 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.874+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe241a02e0 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.875+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7efe14042030 con 0x7efe24101a00 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.876+0000 7efe1a7fc640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7efe100a2b80 0x7efe100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:35.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.876+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7efe140ee890 con 0x7efe24101a00 2024-08-01T04:09:35.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.876+0000 7efe29762640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7efe100a2b80 0x7efe100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:35.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.877+0000 7efe29762640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7efe100a2b80 0x7efe100a5040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe24198b40 tx=0x7efe20005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:09:35.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:35.884+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efe140bc9d0 con 0x7efe24101a00 2024-08-01T04:09:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:35 smithi163 ceph-mon[37383]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:35 smithi163 ceph-mon[37383]: osdmap e27: 4 total, 4 up, 4 in 2024-08-01T04:09:36.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:35 smithi003 ceph-mon[32262]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:36.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:35 smithi003 ceph-mon[32262]: osdmap e27: 4 total, 4 up, 4 in 2024-08-01T04:09:36.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:36.061+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efe241a04b0 con 0x7efe100a2b80 2024-08-01T04:09:36.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:36.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:36.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: osdmap e28: 4 total, 4 up, 4 in 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi003 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:37.893 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:37.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:37.894 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: osdmap e28: 4 total, 4 up, 4 in 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi003 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='client.24233 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: osdmap e28: 4 total, 4 up, 4 in 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi003 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:39.028 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi003 to 8547M 2024-08-01T04:09:39.028 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: osdmap e29: 4 total, 4 up, 4 in 2024-08-01T04:09:39.029 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/1968344322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]: dispatch 2024-08-01T04:09:39.029 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]: dispatch 2024-08-01T04:09:39.029 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]': finished 2024-08-01T04:09:39.029 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: osdmap e30: 5 total, 4 up, 5 in 2024-08-01T04:09:39.029 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:38 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi003 to 8547M 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: osdmap e29: 4 total, 4 up, 4 in 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/1968344322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]: dispatch 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]: dispatch 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]': finished 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: osdmap e30: 5 total, 4 up, 5 in 2024-08-01T04:09:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:38 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:39.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi003 to 8547M 2024-08-01T04:09:39.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: osdmap e29: 4 total, 4 up, 4 in 2024-08-01T04:09:39.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/1968344322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]: dispatch 2024-08-01T04:09:39.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]: dispatch 2024-08-01T04:09:39.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7838d1fc-3355-4659-b61b-5fa5717d70b6"}]': finished 2024-08-01T04:09:39.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: osdmap e30: 5 total, 4 up, 5 in 2024-08-01T04:09:39.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:38 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:39 smithi163 ceph-mon[37383]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:39 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/1087436241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:09:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:39 smithi003 ceph-mon[32262]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:39 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/1087436241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:09:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:39 smithi160 ceph-mon[37224]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:39 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/1087436241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:09:41.986 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:41 smithi160 ceph-mon[37224]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-08-01T04:09:42.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:41 smithi163 ceph-mon[37383]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-08-01T04:09:42.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:41 smithi003 ceph-mon[32262]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-08-01T04:09:43.992 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:43 smithi160 ceph-mon[37224]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:09:44.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:43 smithi163 ceph-mon[37383]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:09:44.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:43 smithi003 ceph-mon[32262]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:09:45.968 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:45 smithi160 ceph-mon[37224]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-01T04:09:45.968 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-01T04:09:45.968 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:46.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:45 smithi163 ceph-mon[37383]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-01T04:09:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-01T04:09:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:46.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:45 smithi003 ceph-mon[32262]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-01T04:09:46.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-01T04:09:46.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:46.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:46 smithi160 ceph-mon[37224]: Deploying daemon osd.4 on smithi160 2024-08-01T04:09:47.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:46 smithi163 ceph-mon[37383]: Deploying daemon osd.4 on smithi160 2024-08-01T04:09:47.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:46 smithi003 ceph-mon[32262]: Deploying daemon osd.4 on smithi160 2024-08-01T04:09:47.760 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:47 smithi160 ceph-mon[37224]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-01T04:09:48.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:47 smithi163 ceph-mon[37383]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-01T04:09:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:47 smithi003 ceph-mon[32262]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-01T04:09:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:49 smithi163 ceph-mon[37383]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-01T04:09:50.046 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:49 smithi160 ceph-mon[37224]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-01T04:09:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:49 smithi003 ceph-mon[32262]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-01T04:09:50.899 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:50.900 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:50.900 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:51.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:51.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:51.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:51.830 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:51 smithi160 ceph-mon[37224]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:52.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:51 smithi163 ceph-mon[37383]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:52.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:51 smithi003 ceph-mon[32262]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:53.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.484+0000 7efe1a7fc640 1 -- 172.21.15.160:0/2929180103 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe241a04b0 con 0x7efe100a2b80 2024-08-01T04:09:53.486 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 4 on host 'smithi160' 2024-08-01T04:09:53.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7efe100a2b80 msgr2=0x7efe100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7efe100a2b80 0x7efe100a5040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efe24198b40 tx=0x7efe20005f90 comp rx=0 tx=0).stop 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 msgr2=0x7efe241981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe241981a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe14009230 tx=0x7efe14031b60 comp rx=0 tx=0).stop 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 shutdown_connections 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7efe100a2b80 0x7efe100a5040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe241023c0 0x7efe2419f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe24101a00 0x7efe241981a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 --2- 172.21.15.160:0/2929180103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe24100800 0x7efe24197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 >> 172.21.15.160:0/2929180103 conn(0x7efe240fbfb0 msgr2=0x7efe24069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 shutdown_connections 2024-08-01T04:09:53.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:09:53.486+0000 7efe2b9c6640 1 -- 172.21.15.160:0/2929180103 wait complete. 2024-08-01T04:09:54.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.218 DEBUG:teuthology.orchestra.run.smithi160:osd.4> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.4.service 2024-08-01T04:09:54.221 INFO:tasks.cephadm:Deploying osd.5 on smithi163 with /dev/vg_nvme/lv_3... 2024-08-01T04:09:54.221 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-01T04:09:54.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:54.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:54.247 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:54.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:54.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.248 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:54.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:54.481 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:09:54.873 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:54 smithi163 ceph-mon[37383]: from='osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-01T04:09:54.874 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:54 smithi163 ceph-mon[37383]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-01T04:09:55.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:54 smithi160 ceph-mon[37224]: from='osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-01T04:09:55.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:54 smithi160 ceph-mon[37224]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-01T04:09:55.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:54 smithi003 ceph-mon[32262]: from='osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-01T04:09:55.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:54 smithi003 ceph-mon[32262]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-01T04:09:55.860 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:55 smithi160 ceph-mon[37224]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:55.860 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:55 smithi160 ceph-mon[37224]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-01T04:09:55.861 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:55 smithi160 ceph-mon[37224]: from='osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:09:55.861 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:55 smithi160 ceph-mon[37224]: osdmap e31: 5 total, 4 up, 5 in 2024-08-01T04:09:55.861 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:55.861 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:55 smithi160 ceph-mon[37224]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:09:55.921 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:55 smithi163 ceph-mon[37383]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:55.921 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:55 smithi163 ceph-mon[37383]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-01T04:09:55.921 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:55 smithi163 ceph-mon[37383]: from='osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:09:55.922 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:55 smithi163 ceph-mon[37383]: osdmap e31: 5 total, 4 up, 5 in 2024-08-01T04:09:55.922 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:55.922 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:55 smithi163 ceph-mon[37383]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:09:56.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:55 smithi003 ceph-mon[32262]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:56.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:55 smithi003 ceph-mon[32262]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-01T04:09:56.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:55 smithi003 ceph-mon[32262]: from='osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:09:56.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:55 smithi003 ceph-mon[32262]: osdmap e31: 5 total, 4 up, 5 in 2024-08-01T04:09:56.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:56.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:55 smithi003 ceph-mon[32262]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:09:56.496 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4[48258]: 2024-08-01T04:09:56.305+0000 7f6c75b05640 -1 osd.4 0 waiting for initial osdmap 2024-08-01T04:09:56.496 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4[48258]: 2024-08-01T04:09:56.311+0000 7f6c6d925640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:09:57.030 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:09:57.055 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_3 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: osdmap e32: 5 total, 4 up, 5 in 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='osd.4 ' entity='osd.4' 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:57.118 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:56 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:57.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: osdmap e32: 5 total, 4 up, 5 in 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='osd.4 ' entity='osd.4' 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:57.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:56 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: osdmap e32: 5 total, 4 up, 5 in 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='osd.4 ' entity='osd.4' 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:57.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:56 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:09:57.352 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:09:57.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 -- 172.21.15.163:0/3205288833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae41075f0 msgr2=0x7f2ae4109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:57.829 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 --2- 172.21.15.163:0/3205288833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae41075f0 0x7f2ae4109a40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009900 tx=0x7f2adc02a410 comp rx=0 tx=0).stop 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 -- 172.21.15.163:0/3205288833 shutdown_connections 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 --2- 172.21.15.163:0/3205288833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae41075f0 0x7f2ae4109a40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 --2- 172.21.15.163:0/3205288833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ae4104cc0 0x7f2ae41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 --2- 172.21.15.163:0/3205288833 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae4102390 0x7f2ae4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 -- 172.21.15.163:0/3205288833 >> 172.21.15.163:0/3205288833 conn(0x7f2ae40fbfb0 msgr2=0x7f2ae40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 -- 172.21.15.163:0/3205288833 shutdown_connections 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 -- 172.21.15.163:0/3205288833 wait complete. 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 Processor -- start 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.802+0000 7f2aea189640 1 -- start start 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ae4102390 0x7f2ae4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:57.830 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 0x7f2ae4100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41075f0 0x7f2ae4100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d660 con 0x7f2ae41075f0 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d4e0 con 0x7f2ae4102390 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d7e0 con 0x7f2ae4104cc0 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 0x7f2ae4100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 0x7f2ae4100340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52048/0 (socket says 172.21.15.163:52048) 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 -- 172.21.15.163:0/2930640360 learned_addr learned my addr 172.21.15.163:0/2930640360 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae37fe640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ae4102390 0x7f2ae4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 -- 172.21.15.163:0/2930640360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ae4102390 msgr2=0x7f2ae4101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ae4102390 0x7f2ae4101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:57.831 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 -- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41075f0 msgr2=0x7f2ae4100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41075f0 0x7f2ae4100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 -- 172.21.15.163:0/2930640360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae4101140 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae2ffd640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 0x7f2ae4100340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005fd0 tx=0x7f2acc00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc017070 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2acc009c30 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41a1df0 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc010460 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.803+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a22a0 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.804+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae40fdaa0 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.806+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2acc003e70 con 0x7f2ae4104cc0 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.806+0000 7f2ae0ff9640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2ad00a2b80 0x7f2ad00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:09:57.832 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.807+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f2acc0cdca0 con 0x7f2ae4104cc0 2024-08-01T04:09:57.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.807+0000 7f2ae37fe640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2ad00a2b80 0x7f2ad00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:09:57.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.808+0000 7f2ae37fe640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2ad00a2b80 0x7f2ad00a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400a400 tx=0x7f2ad400d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:09:57.833 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.811+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2acc020030 con 0x7f2ae4104cc0 2024-08-01T04:09:57.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:09:57.989+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2ae4062360 con 0x7f2ad00a2b80 2024-08-01T04:09:57.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:09:57.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:09:57.990 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi160 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi160 to 8544M 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] boot 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: osdmap e33: 5 total, 5 up, 5 in 2024-08-01T04:09:57.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:57 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi160 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi160 to 8544M 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:58.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:58.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] boot 2024-08-01T04:09:58.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: osdmap e33: 5 total, 5 up, 5 in 2024-08-01T04:09:58.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:57 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:09:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:09:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-08-01T04:09:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi160 2024-08-01T04:09:58.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi160 to 8544M 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: osd.4 [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] boot 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: osdmap e33: 5 total, 5 up, 5 in 2024-08-01T04:09:58.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:57 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:09:59.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:58 smithi163 ceph-mon[37383]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:58 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:58 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:58 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:59.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:58 smithi160 ceph-mon[37224]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:59.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:58 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:59.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:58 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:59.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:58 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:09:59.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:58 smithi003 ceph-mon[32262]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:09:59.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:58 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:09:59.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:58 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:09:59.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:58 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:00.034 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:59 smithi163 ceph-mon[37383]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:00.034 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:09:59 smithi163 ceph-mon[37383]: osdmap e34: 5 total, 5 up, 5 in 2024-08-01T04:10:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:59 smithi160 ceph-mon[37224]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:09:59 smithi160 ceph-mon[37224]: osdmap e34: 5 total, 5 up, 5 in 2024-08-01T04:10:00.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:59 smithi003 ceph-mon[32262]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:00.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:09:59 smithi003 ceph-mon[32262]: osdmap e34: 5 total, 5 up, 5 in 2024-08-01T04:10:01.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:00 smithi160 ceph-mon[37224]: overall HEALTH_OK 2024-08-01T04:10:01.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:00 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/1207083911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]: dispatch 2024-08-01T04:10:01.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:00 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]: dispatch 2024-08-01T04:10:01.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:00 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]': finished 2024-08-01T04:10:01.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:00 smithi160 ceph-mon[37224]: osdmap e35: 6 total, 5 up, 6 in 2024-08-01T04:10:01.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:00 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:00 smithi163 ceph-mon[37383]: overall HEALTH_OK 2024-08-01T04:10:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:00 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/1207083911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]: dispatch 2024-08-01T04:10:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:00 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]: dispatch 2024-08-01T04:10:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:00 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]': finished 2024-08-01T04:10:01.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:00 smithi163 ceph-mon[37383]: osdmap e35: 6 total, 5 up, 6 in 2024-08-01T04:10:01.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:00 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:00 smithi003 ceph-mon[32262]: overall HEALTH_OK 2024-08-01T04:10:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:00 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/1207083911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]: dispatch 2024-08-01T04:10:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:00 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]: dispatch 2024-08-01T04:10:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:00 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f0ce098-7164-49a0-9d74-1c6afc6a32c3"}]': finished 2024-08-01T04:10:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:00 smithi003 ceph-mon[32262]: osdmap e35: 6 total, 5 up, 6 in 2024-08-01T04:10:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:00 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:01 smithi160 ceph-mon[37224]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:01 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/2570174901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:02.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:01 smithi163 ceph-mon[37383]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:02.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:01 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/2570174901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:02.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:01 smithi003 ceph-mon[32262]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:02.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:01 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/2570174901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:04.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:03 smithi160 ceph-mon[37224]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:04.219 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:03 smithi163 ceph-mon[37383]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:04.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:03 smithi003 ceph-mon[32262]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:05.991 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:05 smithi163 ceph-mon[37383]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:05 smithi003 ceph-mon[32262]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:06.228 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:05 smithi160 ceph-mon[37224]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:07 smithi163 ceph-mon[37383]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-01T04:10:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:08.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:07 smithi003 ceph-mon[32262]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:08.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-01T04:10:08.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:08.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:07 smithi160 ceph-mon[37224]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:08.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-01T04:10:08.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:09.191 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:08 smithi163 ceph-mon[37383]: Deploying daemon osd.5 on smithi163 2024-08-01T04:10:09.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:08 smithi003 ceph-mon[32262]: Deploying daemon osd.5 on smithi163 2024-08-01T04:10:09.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:08 smithi160 ceph-mon[37224]: Deploying daemon osd.5 on smithi163 2024-08-01T04:10:10.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:09 smithi163 ceph-mon[37383]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:09 smithi003 ceph-mon[32262]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:10.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:09 smithi160 ceph-mon[37224]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:12.187 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:11 smithi163 ceph-mon[37383]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:12.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:11 smithi003 ceph-mon[32262]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:12.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:11 smithi160 ceph-mon[37224]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:13.226 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:12 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:13.226 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:12 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:13.226 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:12 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:12 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:12 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:12 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:13.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:12 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:13.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:12 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:13.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:12 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:14.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:13 smithi163 ceph-mon[37383]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:14.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:13 smithi003 ceph-mon[32262]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:14.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:13 smithi160 ceph-mon[37224]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:15.573 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.571+0000 7f2ae0ff9640 1 -- 172.21.15.163:0/2930640360 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ae4062360 con 0x7f2ad00a2b80 2024-08-01T04:10:15.573 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 5 on host 'smithi163' 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.573+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2ad00a2b80 msgr2=0x7f2ad00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.573+0000 7f2aea189640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2ad00a2b80 0x7f2ad00a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400a400 tx=0x7f2ad400d040 comp rx=0 tx=0).stop 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.573+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 msgr2=0x7f2ae4100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.573+0000 7f2aea189640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 0x7f2ae4100340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005fd0 tx=0x7f2acc00dde0 comp rx=0 tx=0).stop 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 shutdown_connections 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2ad00a2b80 0x7f2ad00a5040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ae41075f0 0x7f2ae4100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:15.575 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ae4104cc0 0x7f2ae4100340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:15.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 --2- 172.21.15.163:0/2930640360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ae4102390 0x7f2ae4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:15.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 >> 172.21.15.163:0/2930640360 conn(0x7f2ae40fbfb0 msgr2=0x7f2ae4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:10:15.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 shutdown_connections 2024-08-01T04:10:15.576 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:10:15.574+0000 7f2aea189640 1 -- 172.21.15.163:0/2930640360 wait complete. 2024-08-01T04:10:16.226 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:15 smithi163 ceph-mon[37383]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:16.227 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:15 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:16.227 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:15 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:15 smithi003 ceph-mon[32262]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:16.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:15 smithi160 ceph-mon[37224]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:16.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:15 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:16.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:15 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:16.608 DEBUG:teuthology.orchestra.run.smithi163:osd.5> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.5.service 2024-08-01T04:10:16.650 INFO:tasks.cephadm:Deploying osd.6 on smithi003 with /dev/vg_nvme/lv_2... 2024-08-01T04:10:16.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-01T04:10:17.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:16 smithi003 ceph-mon[32262]: from='osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-01T04:10:17.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:16 smithi003 ceph-mon[32262]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-01T04:10:17.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:16 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:17.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:16 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:17.125 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:10:17.221 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:16 smithi160 ceph-mon[37224]: from='osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-01T04:10:17.222 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:16 smithi160 ceph-mon[37224]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-01T04:10:17.222 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:16 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:17.222 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:16 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:17.280 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:16 smithi163 ceph-mon[37383]: from='osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-01T04:10:17.280 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:16 smithi163 ceph-mon[37383]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-01T04:10:17.280 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:16 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:17.280 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:16 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:17 smithi163 ceph-mon[37383]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:17 smithi163 ceph-mon[37383]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-01T04:10:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:17 smithi163 ceph-mon[37383]: osdmap e36: 6 total, 5 up, 6 in 2024-08-01T04:10:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:17 smithi163 ceph-mon[37383]: from='osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:10:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:17 smithi163 ceph-mon[37383]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:10:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:17 smithi003 ceph-mon[32262]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:17 smithi003 ceph-mon[32262]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-01T04:10:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:17 smithi003 ceph-mon[32262]: osdmap e36: 6 total, 5 up, 6 in 2024-08-01T04:10:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:17 smithi003 ceph-mon[32262]: from='osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:10:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:17 smithi003 ceph-mon[32262]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:10:18.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:17 smithi160 ceph-mon[37224]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:18.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:17 smithi160 ceph-mon[37224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-01T04:10:18.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:17 smithi160 ceph-mon[37224]: osdmap e36: 6 total, 5 up, 6 in 2024-08-01T04:10:18.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:18.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:17 smithi160 ceph-mon[37224]: from='osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:10:18.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:17 smithi160 ceph-mon[37224]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:10:18.641 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5[49143]: 2024-08-01T04:10:18.418+0000 7fa2cffb6640 -1 osd.5 0 waiting for initial osdmap 2024-08-01T04:10:18.642 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5[49143]: 2024-08-01T04:10:18.424+0000 7fa2c7dd6640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: osdmap e37: 6 total, 5 up, 6 in 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: from='osd.5 ' entity='osd.5' 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] boot 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: osdmap e38: 6 total, 6 up, 6 in 2024-08-01T04:10:19.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.238 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: osdmap e37: 6 total, 5 up, 6 in 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: from='osd.5 ' entity='osd.5' 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] boot 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: osdmap e38: 6 total, 6 up, 6 in 2024-08-01T04:10:19.239 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:10:19.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: osdmap e37: 6 total, 5 up, 6 in 2024-08-01T04:10:19.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:19.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: from='osd.5 ' entity='osd.5' 2024-08-01T04:10:19.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: osd.5 [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] boot 2024-08-01T04:10:19.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: osdmap e38: 6 total, 6 up, 6 in 2024-08-01T04:10:19.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:10:20.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.010 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:20.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:20.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:19 smithi003 ceph-mon[32262]: osdmap e39: 6 total, 6 up, 6 in 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:20.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:19 smithi163 ceph-mon[37383]: osdmap e39: 6 total, 6 up, 6 in 2024-08-01T04:10:20.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:10:20.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:19 smithi160 ceph-mon[37224]: osdmap e39: 6 total, 6 up, 6 in 2024-08-01T04:10:20.693 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:10:20.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_2 2024-08-01T04:10:21.175 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi163 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi163 to 8544M 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:20 smithi163 ceph-mon[37383]: osdmap e40: 6 total, 6 up, 6 in 2024-08-01T04:10:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi163 2024-08-01T04:10:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi163 to 8544M 2024-08-01T04:10:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:20 smithi003 ceph-mon[32262]: osdmap e40: 6 total, 6 up, 6 in 2024-08-01T04:10:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi163 2024-08-01T04:10:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi163 to 8544M 2024-08-01T04:10:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:21.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:21.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:21.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:20 smithi160 ceph-mon[37224]: osdmap e40: 6 total, 6 up, 6 in 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.786+0000 7fbe3fc00640 1 -- 172.21.15.3:0/559293649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 msgr2=0x7fbe381047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.786+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/559293649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe381047b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009900 tx=0x7fbe2c02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 -- 172.21.15.3:0/559293649 shutdown_connections 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/559293649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe38107620 0x7fbe38109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/559293649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38104cf0 0x7fbe381070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/559293649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe381047b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:21.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 -- 172.21.15.3:0/559293649 >> 172.21.15.3:0/559293649 conn(0x7fbe380fbfa0 msgr2=0x7fbe380fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:10:21.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 -- 172.21.15.3:0/559293649 shutdown_connections 2024-08-01T04:10:21.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.787+0000 7fbe3fc00640 1 -- 172.21.15.3:0/559293649 wait complete. 2024-08-01T04:10:21.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.788+0000 7fbe3fc00640 1 Processor -- start 2024-08-01T04:10:21.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.788+0000 7fbe3fc00640 1 -- start start 2024-08-01T04:10:21.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3fc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe38101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3fc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe38104cf0 0x7fbe38100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3fc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38107620 0x7fbe381008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3fc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810d620 con 0x7fbe38107620 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3fc00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810d4a0 con 0x7fbe38104cf0 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3fc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3810d7a0 con 0x7fbe381023c0 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe38101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe38101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:39850/0 (socket says 172.21.15.3:39850) 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3d200640 1 -- 172.21.15.3:0/1604458493 learned_addr learned my addr 172.21.15.3:0/1604458493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe3dc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38107620 0x7fbe381008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.789+0000 7fbe37e00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe38104cf0 0x7fbe38100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe3d200640 1 -- 172.21.15.3:0/1604458493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe38104cf0 msgr2=0x7fbe38100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:21.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe3d200640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe38104cf0 0x7fbe38100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:21.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe3d200640 1 -- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38107620 msgr2=0x7fbe381008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:21.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe3d200640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38107620 0x7fbe381008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:21.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe3d200640 1 -- 172.21.15.3:0/1604458493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe38101170 con 0x7fbe381023c0 2024-08-01T04:10:21.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe3d200640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe38101d20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0314e0 tx=0x7fbe2c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:10:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.790+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c03f030 con 0x7fbe381023c0 2024-08-01T04:10:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.791+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2c003ce0 con 0x7fbe381023c0 2024-08-01T04:10:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.791+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe381a1e30 con 0x7fbe381023c0 2024-08-01T04:10:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.791+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c035070 con 0x7fbe381023c0 2024-08-01T04:10:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.791+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381a21d0 con 0x7fbe381023c0 2024-08-01T04:10:21.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.792+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe380fda90 con 0x7fbe381023c0 2024-08-01T04:10:21.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.794+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbe2c042030 con 0x7fbe381023c0 2024-08-01T04:10:21.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.795+0000 7fbe35600640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe200a2b80 0x7fbe200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.796+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fbe2c0eea90 con 0x7fbe381023c0 2024-08-01T04:10:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.796+0000 7fbe37e00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe200a2b80 0x7fbe200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:21.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.797+0000 7fbe37e00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe200a2b80 0x7fbe200a5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe38101aa0 tx=0x7fbe2400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:10:21.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:21.797+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbe2c0bc1a0 con 0x7fbe381023c0 2024-08-01T04:10:22.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:22.001+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbe38062360 con 0x7fbe200a2b80 2024-08-01T04:10:22.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:21 smithi163 ceph-mon[37383]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:22.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:21 smithi003 ceph-mon[32262]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:22.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:21 smithi160 ceph-mon[37224]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:23.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:22 smithi003 ceph-mon[32262]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:10:23.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:22 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:10:23.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:22 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:10:23.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:22 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:23.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:22 smithi003 ceph-mon[32262]: osdmap e41: 6 total, 6 up, 6 in 2024-08-01T04:10:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:22 smithi163 ceph-mon[37383]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:10:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:22 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:10:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:22 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:10:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:22 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:22 smithi163 ceph-mon[37383]: osdmap e41: 6 total, 6 up, 6 in 2024-08-01T04:10:23.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:22 smithi160 ceph-mon[37224]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:10:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:22 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:10:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:22 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:10:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:22 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:22 smithi160 ceph-mon[37224]: osdmap e41: 6 total, 6 up, 6 in 2024-08-01T04:10:24.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:23 smithi003 ceph-mon[32262]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:24.205 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:23 smithi003 ceph-mon[32262]: osdmap e42: 6 total, 6 up, 6 in 2024-08-01T04:10:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:23 smithi163 ceph-mon[37383]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:23 smithi163 ceph-mon[37383]: osdmap e42: 6 total, 6 up, 6 in 2024-08-01T04:10:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:23 smithi160 ceph-mon[37224]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:23 smithi160 ceph-mon[37224]: osdmap e42: 6 total, 6 up, 6 in 2024-08-01T04:10:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:26 smithi163 ceph-mon[37383]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-08-01T04:10:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:26 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3749982818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]: dispatch 2024-08-01T04:10:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:26 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]: dispatch 2024-08-01T04:10:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:26 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]': finished 2024-08-01T04:10:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:26 smithi163 ceph-mon[37383]: osdmap e43: 7 total, 6 up, 7 in 2024-08-01T04:10:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:26 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:26 smithi003 ceph-mon[32262]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-08-01T04:10:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:26 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3749982818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]: dispatch 2024-08-01T04:10:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:26 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]: dispatch 2024-08-01T04:10:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:26 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]': finished 2024-08-01T04:10:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:26 smithi003 ceph-mon[32262]: osdmap e43: 7 total, 6 up, 7 in 2024-08-01T04:10:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:26 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:26.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:26 smithi160 ceph-mon[37224]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-08-01T04:10:26.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:26 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3749982818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]: dispatch 2024-08-01T04:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:26 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]: dispatch 2024-08-01T04:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:26 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3de683d-970d-4e25-9574-48cf63d401ae"}]': finished 2024-08-01T04:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:26 smithi160 ceph-mon[37224]: osdmap e43: 7 total, 6 up, 7 in 2024-08-01T04:10:26.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:26 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:27.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:27 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/4086336264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:27.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:27 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/4086336264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:27.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:27 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/4086336264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:28.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:28 smithi163 ceph-mon[37383]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-08-01T04:10:28.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:28 smithi003 ceph-mon[32262]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-08-01T04:10:28.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:28 smithi160 ceph-mon[37224]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-08-01T04:10:29.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:29 smithi163 ceph-mon[37383]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:10:29.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:29 smithi003 ceph-mon[32262]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:10:29.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:29 smithi160 ceph-mon[37224]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:10:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:31 smithi163 ceph-mon[37383]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-01T04:10:31.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:31 smithi003 ceph-mon[32262]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-01T04:10:31.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:31 smithi160 ceph-mon[37224]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-01T04:10:33.341 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:33 smithi003 ceph-mon[32262]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-08-01T04:10:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:33 smithi163 ceph-mon[37383]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-08-01T04:10:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:33 smithi160 ceph-mon[37224]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-08-01T04:10:35.455 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:35 smithi160 ceph-mon[37224]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:35.455 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-01T04:10:35.455 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:35.455 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:35 smithi160 ceph-mon[37224]: Deploying daemon osd.6 on smithi003 2024-08-01T04:10:35.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:35 smithi163 ceph-mon[37383]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:35.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-01T04:10:35.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:35.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:35 smithi163 ceph-mon[37383]: Deploying daemon osd.6 on smithi003 2024-08-01T04:10:35.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:35 smithi003 ceph-mon[32262]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:35.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-01T04:10:35.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:35.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:35 smithi003 ceph-mon[32262]: Deploying daemon osd.6 on smithi003 2024-08-01T04:10:37.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:37 smithi003 ceph-mon[32262]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:37.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:37 smithi163 ceph-mon[37383]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:37.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:37 smithi160 ceph-mon[37224]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:39.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:39 smithi163 ceph-mon[37383]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:39.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:39 smithi003 ceph-mon[32262]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:39.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:39 smithi160 ceph-mon[37224]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:41.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:41 smithi163 ceph-mon[37383]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:41.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:41 smithi003 ceph-mon[32262]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:41.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:41 smithi160 ceph-mon[37224]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:43 smithi003 ceph-mon[32262]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:43.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:43 smithi163 ceph-mon[37383]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:43.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:43.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:43.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:43 smithi160 ceph-mon[37224]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:45.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:45 smithi003 ceph-mon[32262]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:45.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:45 smithi163 ceph-mon[37383]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:45.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:45 smithi160 ceph-mon[37224]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:47.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.770+0000 7fbe35600640 1 -- 172.21.15.3:0/1604458493 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbe38062360 con 0x7fbe200a2b80 2024-08-01T04:10:47.774 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 6 on host 'smithi003' 2024-08-01T04:10:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe200a2b80 msgr2=0x7fbe200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe200a2b80 0x7fbe200a5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe38101aa0 tx=0x7fbe2400b390 comp rx=0 tx=0).stop 2024-08-01T04:10:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 msgr2=0x7fbe38101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe38101d20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0314e0 tx=0x7fbe2c032ce0 comp rx=0 tx=0).stop 2024-08-01T04:10:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 shutdown_connections 2024-08-01T04:10:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe200a2b80 0x7fbe200a5040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe38107620 0x7fbe381008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.773+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe38104cf0 0x7fbe38100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.774+0000 7fbe3fc00640 1 --2- 172.21.15.3:0/1604458493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe381023c0 0x7fbe38101d20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.774+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 >> 172.21.15.3:0/1604458493 conn(0x7fbe380fbfa0 msgr2=0x7fbe38102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:10:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.774+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 shutdown_connections 2024-08-01T04:10:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:10:47.774+0000 7fbe3fc00640 1 -- 172.21.15.3:0/1604458493 wait complete. 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:47 smithi163 ceph-mon[37383]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-01T04:10:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:48.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:47 smithi003 ceph-mon[32262]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:47 smithi160 ceph-mon[37224]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-01T04:10:48.678 DEBUG:teuthology.orchestra.run.smithi003:osd.6> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.6.service 2024-08-01T04:10:48.684 INFO:tasks.cephadm:Deploying osd.7 on smithi160 with /dev/vg_nvme/lv_2... 2024-08-01T04:10:48.684 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-01T04:10:48.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-01T04:10:48.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: osdmap e44: 7 total, 6 up, 7 in 2024-08-01T04:10:48.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:10:48.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:48.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:48.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:48.946 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: osdmap e44: 7 total, 6 up, 7 in 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:49.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:49.069 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-01T04:10:49.069 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: osdmap e44: 7 total, 6 up, 7 in 2024-08-01T04:10:49.069 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:10:49.069 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:49.069 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:10:49.069 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:49.070 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:49 smithi003 ceph-mon[32262]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:49 smithi003 ceph-mon[32262]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:10:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:49 smithi003 ceph-mon[32262]: osdmap e45: 7 total, 6 up, 7 in 2024-08-01T04:10:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:49.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:49 smithi163 ceph-mon[37383]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:49 smithi163 ceph-mon[37383]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:10:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:49 smithi163 ceph-mon[37383]: osdmap e45: 7 total, 6 up, 7 in 2024-08-01T04:10:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:50.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:49 smithi160 ceph-mon[37224]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:50.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:49 smithi160 ceph-mon[37224]: from='osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:10:50.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:49 smithi160 ceph-mon[37224]: osdmap e45: 7 total, 6 up, 7 in 2024-08-01T04:10:50.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:50.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:50 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:10:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:50 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:10:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:50 smithi163 ceph-mon[37383]: osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] boot 2024-08-01T04:10:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:50 smithi163 ceph-mon[37383]: osdmap e46: 7 total, 7 up, 7 in 2024-08-01T04:10:51.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:51.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:50 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:10:51.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:50 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:10:51.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:50 smithi003 ceph-mon[32262]: osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] boot 2024-08-01T04:10:51.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:50 smithi003 ceph-mon[32262]: osdmap e46: 7 total, 7 up, 7 in 2024-08-01T04:10:51.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:51.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:50 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:10:51.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:50 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:10:51.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:50 smithi160 ceph-mon[37224]: osd.6 [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] boot 2024-08-01T04:10:51.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:50 smithi160 ceph-mon[37224]: osdmap e46: 7 total, 7 up, 7 in 2024-08-01T04:10:51.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:10:51.519 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:10:51.545 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-08-01T04:10:51.805 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:10:51.821 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:51 smithi160 ceph-mon[37224]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:51.821 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:51 smithi160 ceph-mon[37224]: osdmap e47: 7 total, 7 up, 7 in 2024-08-01T04:10:52.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:51 smithi163 ceph-mon[37383]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:52.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:51 smithi163 ceph-mon[37383]: osdmap e47: 7 total, 7 up, 7 in 2024-08-01T04:10:52.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:51 smithi003 ceph-mon[32262]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-01T04:10:52.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:51 smithi003 ceph-mon[32262]: osdmap e47: 7 total, 7 up, 7 in 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 -- 172.21.15.160:0/106404618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 msgr2=0x7f1818101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 --2- 172.21.15.160:0/106404618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818101e80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f180c009900 tx=0x7f180c02a410 comp rx=0 tx=0).stop 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 -- 172.21.15.160:0/106404618 shutdown_connections 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 --2- 172.21.15.160:0/106404618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18181023c0 0x7f1818108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 --2- 172.21.15.160:0/106404618 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818101e80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 --2- 172.21.15.160:0/106404618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1818100800 0x7f1818100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 -- 172.21.15.160:0/106404618 >> 172.21.15.160:0/106404618 conn(0x7f18180fbfb0 msgr2=0x7f18180fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 -- 172.21.15.160:0/106404618 shutdown_connections 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 -- 172.21.15.160:0/106404618 wait complete. 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.236+0000 7f181d3e7640 1 Processor -- start 2024-08-01T04:10:52.253 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 -- start start 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1818100800 0x7f1818104920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18181023c0 0x7f18181034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f181810c500 con 0x7f1818100800 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f181810c380 con 0x7f1818101a00 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f181d3e7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f181810c680 con 0x7f18181023c0 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:52596/0 (socket says 172.21.15.160:52596) 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 -- 172.21.15.160:0/1957053637 learned_addr learned my addr 172.21.15.160:0/1957053637 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18177fe640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18181023c0 0x7f18181034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 -- 172.21.15.160:0/1957053637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18181023c0 msgr2=0x7f18181034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:52.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18181023c0 0x7f18181034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 -- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1818100800 msgr2=0x7f1818104920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1818100800 0x7f1818104920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 -- 172.21.15.160:0/1957053637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1818103d70 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.237+0000 7f18167fc640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818102f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f180c009230 tx=0x7f180c004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.238+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c03f030 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.238+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18181a1e20 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.238+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c003ce0 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.238+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c035070 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.238+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181a22f0 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.239+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18181a2520 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.240+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f180c042030 con 0x7f1818101a00 2024-08-01T04:10:52.255 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.240+0000 7f17fffff640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f18000a2b80 0x7f18000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:10:52.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.240+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f180c0eef30 con 0x7f1818101a00 2024-08-01T04:10:52.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.241+0000 7f1816ffd640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f18000a2b80 0x7f18000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:10:52.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.242+0000 7f1816ffd640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f18000a2b80 0x7f18000a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1818101860 tx=0x7f1808006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:10:52.256 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.244+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f180c0bcba0 con 0x7f1818101a00 2024-08-01T04:10:52.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:10:52.425+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f181806a3b0 con 0x7f18000a2b80 2024-08-01T04:10:53.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:10:53.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:10:53.006 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:53.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:10:53.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:10:53.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:53.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:10:53.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:10:53.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:10:54.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:10:55.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:54 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi003 2024-08-01T04:10:55.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:54 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi003 to 5698M 2024-08-01T04:10:55.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:54 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi003 2024-08-01T04:10:55.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:54 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi003 to 5698M 2024-08-01T04:10:55.055 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:54 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi003 2024-08-01T04:10:55.055 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:54 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi003 to 5698M 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/4008473118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]: dispatch 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]: dispatch 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]': finished 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: osdmap e48: 8 total, 7 up, 8 in 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:10:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:55 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/3113070388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:56.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:56.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/4008473118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]: dispatch 2024-08-01T04:10:56.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]: dispatch 2024-08-01T04:10:56.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]': finished 2024-08-01T04:10:56.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: osdmap e48: 8 total, 7 up, 8 in 2024-08-01T04:10:56.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:10:56.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:55 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/3113070388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:56.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/4008473118' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]: dispatch 2024-08-01T04:10:56.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]: dispatch 2024-08-01T04:10:56.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2878577-9071-4b13-92ed-0cace6099cfd"}]': finished 2024-08-01T04:10:56.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: osdmap e48: 8 total, 7 up, 8 in 2024-08-01T04:10:56.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:10:56.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:55 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/3113070388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:10:58.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:57 smithi163 ceph-mon[37383]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:57 smithi003 ceph-mon[32262]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:10:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:57 smithi160 ceph-mon[37224]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:00.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:10:59 smithi163 ceph-mon[37383]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:00.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:10:59 smithi003 ceph-mon[32262]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:10:59 smithi160 ceph-mon[37224]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:02.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:01 smithi163 ceph-mon[37383]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:02.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-01T04:11:02.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:02.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:01 smithi003 ceph-mon[32262]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:02.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-01T04:11:02.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:01 smithi160 ceph-mon[37224]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-01T04:11:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:03.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:02 smithi163 ceph-mon[37383]: Deploying daemon osd.7 on smithi160 2024-08-01T04:11:03.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:02 smithi003 ceph-mon[32262]: Deploying daemon osd.7 on smithi160 2024-08-01T04:11:03.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:02 smithi160 ceph-mon[37224]: Deploying daemon osd.7 on smithi160 2024-08-01T04:11:04.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:03 smithi163 ceph-mon[37383]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:04.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:03 smithi003 ceph-mon[32262]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:04.138 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:03 smithi160 ceph-mon[37224]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:05 smithi163 ceph-mon[37383]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:05 smithi003 ceph-mon[32262]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:05 smithi160 ceph-mon[37224]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:08.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:07 smithi003 ceph-mon[32262]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:08.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:07 smithi160 ceph-mon[37224]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:08.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:08.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:08.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:07 smithi163 ceph-mon[37383]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.145 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:09 smithi160 ceph-mon[37224]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:10.145 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.145 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.146 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:10.146 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:10.146 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:09 smithi163 ceph-mon[37383]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:09 smithi003 ceph-mon[32262]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:10.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.508+0000 7f17fffff640 1 -- 172.21.15.160:0/1957053637 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f181806a3b0 con 0x7f18000a2b80 2024-08-01T04:11:10.511 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 7 on host 'smithi160' 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f18000a2b80 msgr2=0x7f18000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f18000a2b80 0x7f18000a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1818101860 tx=0x7f1808006c90 comp rx=0 tx=0).stop 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 msgr2=0x7f1818102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818102f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f180c009230 tx=0x7f180c004430 comp rx=0 tx=0).stop 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 shutdown_connections 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f18000a2b80 0x7f18000a5040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18181023c0 0x7f18181034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1818101a00 0x7f1818102f70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 --2- 172.21.15.160:0/1957053637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1818100800 0x7f1818104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 >> 172.21.15.160:0/1957053637 conn(0x7f18180fbfb0 msgr2=0x7f1818069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 shutdown_connections 2024-08-01T04:11:10.513 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:11:10.511+0000 7f181d3e7640 1 -- 172.21.15.160:0/1957053637 wait complete. 2024-08-01T04:11:11.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:11.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:11.091 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:11.118 DEBUG:teuthology.orchestra.run.smithi160:osd.7> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.7.service 2024-08-01T04:11:11.121 INFO:tasks.cephadm:Deploying osd.8 on smithi163 with /dev/vg_nvme/lv_2... 2024-08-01T04:11:11.121 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-01T04:11:11.147 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:11.147 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:11.147 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:11.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:11.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:11.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:11.381 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:11:12.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:11 smithi163 ceph-mon[37383]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:12.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:11 smithi163 ceph-mon[37383]: from='osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-01T04:11:12.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:11 smithi163 ceph-mon[37383]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-01T04:11:12.056 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:11 smithi160 ceph-mon[37224]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:12.056 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:11 smithi160 ceph-mon[37224]: from='osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-01T04:11:12.056 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:11 smithi160 ceph-mon[37224]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-01T04:11:12.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:11 smithi003 ceph-mon[32262]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:12.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:11 smithi003 ceph-mon[32262]: from='osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-01T04:11:12.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:11 smithi003 ceph-mon[32262]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-01T04:11:12.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-mon[37224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-01T04:11:12.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-mon[37224]: from='osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:11:12.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-mon[37224]: osdmap e49: 8 total, 7 up, 8 in 2024-08-01T04:11:12.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:12.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-mon[37224]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:11:12.903 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7[54715]: 2024-08-01T04:11:12.837+0000 7fdb98117640 -1 osd.7 0 waiting for initial osdmap 2024-08-01T04:11:12.903 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:11:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7[54715]: 2024-08-01T04:11:12.848+0000 7fdb93f3f640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:11:13.128 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:12 smithi163 ceph-mon[37383]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-01T04:11:13.129 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:12 smithi163 ceph-mon[37383]: from='osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:11:13.129 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:12 smithi163 ceph-mon[37383]: osdmap e49: 8 total, 7 up, 8 in 2024-08-01T04:11:13.129 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:12 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:13.129 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:12 smithi163 ceph-mon[37383]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:11:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:12 smithi003 ceph-mon[32262]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-01T04:11:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:12 smithi003 ceph-mon[32262]: from='osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:11:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:12 smithi003 ceph-mon[32262]: osdmap e49: 8 total, 7 up, 8 in 2024-08-01T04:11:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:12 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:12 smithi003 ceph-mon[32262]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:11:14.023 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:14.023 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:11:14.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: osdmap e50: 8 total, 7 up, 8 in 2024-08-01T04:11:14.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] boot 2024-08-01T04:11:14.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: osdmap e51: 8 total, 8 up, 8 in 2024-08-01T04:11:14.024 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.032 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: osdmap e50: 8 total, 7 up, 8 in 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] boot 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: osdmap e51: 8 total, 8 up, 8 in 2024-08-01T04:11:14.033 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.036 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:11:14.063 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_2 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: osdmap e50: 8 total, 7 up, 8 in 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: osd.7 [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] boot 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: osdmap e51: 8 total, 8 up, 8 in 2024-08-01T04:11:14.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:11:14.369 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:11:14.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 -- 172.21.15.163:0/1498272374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 msgr2=0x7fe2f4107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:14.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/1498272374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f4107160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009900 tx=0x7fe2e402a410 comp rx=0 tx=0).stop 2024-08-01T04:11:14.892 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 -- 172.21.15.163:0/1498272374 shutdown_connections 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/1498272374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f41076a0 0x7fe2f4109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/1498272374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f4107160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/1498272374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f4102440 0x7fe2f4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 -- 172.21.15.163:0/1498272374 >> 172.21.15.163:0/1498272374 conn(0x7fe2f40fbfc0 msgr2=0x7fe2f40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 -- 172.21.15.163:0/1498272374 shutdown_connections 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 -- 172.21.15.163:0/1498272374 wait complete. 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.886+0000 7fe2fb78a640 1 Processor -- start 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 -- start start 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f4102440 0x7fe2f4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:14.893 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f41076a0 0x7fe2f419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410d630 con 0x7fe2f41076a0 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410d4b0 con 0x7fe2f4102440 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410d7b0 con 0x7fe2f4104d70 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42188/0 (socket says 172.21.15.163:42188) 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 -- 172.21.15.163:0/275331188 learned_addr learned my addr 172.21.15.163:0/275331188 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f9526640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f4102440 0x7fe2f4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:14.894 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 -- 172.21.15.163:0/275331188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f4102440 msgr2=0x7fe2f4197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f4102440 0x7fe2f4197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 -- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f41076a0 msgr2=0x7fe2f419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f41076a0 0x7fe2f419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 -- 172.21.15.163:0/275331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f419f9d0 con 0x7fe2f4104d70 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2f8d25640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f41982e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4031610 tx=0x7fe2e40043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e403f030 con 0x7fe2f4104d70 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2e4003ce0 con 0x7fe2f4104d70 2024-08-01T04:11:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.887+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f419fc60 con 0x7fe2f4104d70 2024-08-01T04:11:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.888+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4035070 con 0x7fe2f4104d70 2024-08-01T04:11:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.888+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41a0190 con 0x7fe2f4104d70 2024-08-01T04:11:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.889+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f40fdab0 con 0x7fe2f4104d70 2024-08-01T04:11:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.890+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe2e4042030 con 0x7fe2f4104d70 2024-08-01T04:11:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.890+0000 7fe2ea7fc640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe2e00a2c30 0x7fe2e00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.890+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5426+0+0 (secure 0 0 0) 0x7fe2e40ee9e0 con 0x7fe2f4104d70 2024-08-01T04:11:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.891+0000 7fe2f9526640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe2e00a2c30 0x7fe2e00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:14.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.892+0000 7fe2f9526640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe2e00a2c30 0x7fe2e00a50f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009990 tx=0x7fe2f0009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:11:14.898 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:14.895+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe2e40bc480 con 0x7fe2f4104d70 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:15.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:15.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:15.070+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe2f4062360 con 0x7fe2e00a2c30 2024-08-01T04:11:15.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:15.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:16.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi160 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi160 to 5696M 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: osdmap e52: 8 total, 8 up, 8 in 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:15 smithi160 ceph-mon[37224]: osdmap e53: 8 total, 8 up, 8 in 2024-08-01T04:11:16.200 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi160 2024-08-01T04:11:16.200 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi160 to 5696M 2024-08-01T04:11:16.200 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:16.201 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: osdmap e52: 8 total, 8 up, 8 in 2024-08-01T04:11:16.201 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:11:16.201 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:11:16.201 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:16.201 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:15 smithi163 ceph-mon[37383]: osdmap e53: 8 total, 8 up, 8 in 2024-08-01T04:11:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi160 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi160 to 5696M 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: pgmap v174: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: osdmap e52: 8 total, 8 up, 8 in 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:15 smithi003 ceph-mon[32262]: osdmap e53: 8 total, 8 up, 8 in 2024-08-01T04:11:17.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:16 smithi160 ceph-mon[37224]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:11:17.157 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:16 smithi163 ceph-mon[37383]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:11:17.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:16 smithi003 ceph-mon[32262]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: osdmap e54: 8 total, 8 up, 8 in 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/4144383108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]: dispatch 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]: dispatch 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]': finished 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: osdmap e55: 9 total, 8 up, 9 in 2024-08-01T04:11:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: osdmap e54: 8 total, 8 up, 8 in 2024-08-01T04:11:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/4144383108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]: dispatch 2024-08-01T04:11:18.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]: dispatch 2024-08-01T04:11:18.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]': finished 2024-08-01T04:11:18.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: osdmap e55: 9 total, 8 up, 9 in 2024-08-01T04:11:18.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: osdmap e54: 8 total, 8 up, 8 in 2024-08-01T04:11:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/4144383108' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]: dispatch 2024-08-01T04:11:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]: dispatch 2024-08-01T04:11:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c865f68f-ddd2-4332-bbfc-26b68a4b85b8"}]': finished 2024-08-01T04:11:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: osdmap e55: 9 total, 8 up, 9 in 2024-08-01T04:11:18.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:19.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:18 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/3948230408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:11:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:18 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/3948230408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:11:19.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:18 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/3948230408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:11:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:19 smithi160 ceph-mon[37224]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:19 smithi160 ceph-mon[37224]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-08-01T04:11:20.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:19 smithi163 ceph-mon[37383]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:19 smithi163 ceph-mon[37383]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-08-01T04:11:20.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:19 smithi003 ceph-mon[32262]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:19 smithi003 ceph-mon[32262]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-08-01T04:11:21.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:20 smithi163 ceph-mon[37383]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-08-01T04:11:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:20 smithi163 ceph-mon[37383]: Cluster is now healthy 2024-08-01T04:11:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:20 smithi003 ceph-mon[32262]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-08-01T04:11:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:20 smithi003 ceph-mon[32262]: Cluster is now healthy 2024-08-01T04:11:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:20 smithi160 ceph-mon[37224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-08-01T04:11:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:20 smithi160 ceph-mon[37224]: Cluster is now healthy 2024-08-01T04:11:22.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:21 smithi163 ceph-mon[37383]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:22.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:21 smithi003 ceph-mon[32262]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:22.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:21 smithi160 ceph-mon[37224]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:24.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:23 smithi163 ceph-mon[37383]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:24.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:23 smithi003 ceph-mon[32262]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:23 smithi160 ceph-mon[37224]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:25.042 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-01T04:11:25.042 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-01T04:11:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-01T04:11:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:26.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:25 smithi163 ceph-mon[37383]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:26.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:25 smithi163 ceph-mon[37383]: Deploying daemon osd.8 on smithi163 2024-08-01T04:11:26.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:25 smithi003 ceph-mon[32262]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:26.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:25 smithi003 ceph-mon[32262]: Deploying daemon osd.8 on smithi163 2024-08-01T04:11:26.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:25 smithi160 ceph-mon[37224]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:26.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:25 smithi160 ceph-mon[37224]: Deploying daemon osd.8 on smithi163 2024-08-01T04:11:28.016 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:27 smithi163 ceph-mon[37383]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:28.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:27 smithi003 ceph-mon[32262]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:28.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:27 smithi160 ceph-mon[37224]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:30.277 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:29 smithi163 ceph-mon[37383]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:30.277 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:30.277 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:30.278 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:30.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:29 smithi003 ceph-mon[32262]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:30.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:29 smithi160 ceph-mon[37224]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:30.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:30.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:30.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:32.263 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:31 smithi163 ceph-mon[37383]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:32.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:31 smithi003 ceph-mon[32262]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:32.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:31 smithi160 ceph-mon[37224]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:32.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.985+0000 7fe2ea7fc640 1 -- 172.21.15.163:0/275331188 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe2f4062360 con 0x7fe2e00a2c30 2024-08-01T04:11:32.987 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 8 on host 'smithi163' 2024-08-01T04:11:32.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe2e00a2c30 msgr2=0x7fe2e00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe2e00a2c30 0x7fe2e00a50f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009990 tx=0x7fe2f0009110 comp rx=0 tx=0).stop 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 msgr2=0x7fe2f41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f41982e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4031610 tx=0x7fe2e40043b0 comp rx=0 tx=0).stop 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 shutdown_connections 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe2e00a2c30 0x7fe2e00a50f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2f41076a0 0x7fe2f419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f4104d70 0x7fe2f41982e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 --2- 172.21.15.163:0/275331188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe2f4102440 0x7fe2f4197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 >> 172.21.15.163:0/275331188 conn(0x7fe2f40fbfc0 msgr2=0x7fe2f4104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 shutdown_connections 2024-08-01T04:11:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:11:32.987+0000 7fe2fb78a640 1 -- 172.21.15.163:0/275331188 wait complete. 2024-08-01T04:11:33.634 DEBUG:teuthology.orchestra.run.smithi163:osd.8> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.8.service 2024-08-01T04:11:33.636 INFO:tasks.cephadm:Deploying osd.9 on smithi003 with /dev/vg_nvme/lv_1... 2024-08-01T04:11:33.637 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-01T04:11:34.099 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: from='osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:33 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: from='osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:33 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: from='osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-01T04:11:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-01T04:11:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:33 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:35.121 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:34 smithi163 ceph-mon[37383]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-01T04:11:35.121 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:34 smithi163 ceph-mon[37383]: from='osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:11:35.122 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:34 smithi163 ceph-mon[37383]: osdmap e56: 9 total, 8 up, 9 in 2024-08-01T04:11:35.122 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:34 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:35.122 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:34 smithi163 ceph-mon[37383]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:11:35.256 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:34 smithi003 ceph-mon[32262]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-01T04:11:35.257 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:34 smithi003 ceph-mon[32262]: from='osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:11:35.257 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:34 smithi003 ceph-mon[32262]: osdmap e56: 9 total, 8 up, 9 in 2024-08-01T04:11:35.257 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:34 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:35.257 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:34 smithi003 ceph-mon[32262]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:11:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:34 smithi160 ceph-mon[37224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-01T04:11:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:34 smithi160 ceph-mon[37224]: from='osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:11:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:34 smithi160 ceph-mon[37224]: osdmap e56: 9 total, 8 up, 9 in 2024-08-01T04:11:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:34 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:34 smithi160 ceph-mon[37224]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:11:35.692 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:11:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8[55910]: 2024-08-01T04:11:35.406+0000 7f79a8d5d640 -1 osd.8 0 waiting for initial osdmap 2024-08-01T04:11:35.692 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:11:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8[55910]: 2024-08-01T04:11:35.417+0000 7f79a0b7d640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:11:36.118 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:36 smithi003 ceph-mon[32262]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:36.118 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:36 smithi003 ceph-mon[32262]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:11:36.118 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:36 smithi003 ceph-mon[32262]: osdmap e57: 9 total, 8 up, 9 in 2024-08-01T04:11:36.118 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:36.118 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:36 smithi003 ceph-mon[32262]: from='osd.8 ' entity='osd.8' 2024-08-01T04:11:36.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:36 smithi163 ceph-mon[37383]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:36 smithi163 ceph-mon[37383]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:11:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:36 smithi163 ceph-mon[37383]: osdmap e57: 9 total, 8 up, 9 in 2024-08-01T04:11:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:36 smithi163 ceph-mon[37383]: from='osd.8 ' entity='osd.8' 2024-08-01T04:11:36.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:36 smithi160 ceph-mon[37224]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-08-01T04:11:36.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:36 smithi160 ceph-mon[37224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:11:36.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:36 smithi160 ceph-mon[37224]: osdmap e57: 9 total, 8 up, 9 in 2024-08-01T04:11:36.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:36.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:36 smithi160 ceph-mon[37224]: from='osd.8 ' entity='osd.8' 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] boot 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: osdmap e58: 9 total, 9 up, 9 in 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:37.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:37.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:37 smithi003 ceph-mon[32262]: osdmap e59: 9 total, 9 up, 9 in 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] boot 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: osdmap e58: 9 total, 9 up, 9 in 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:37.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:37.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:37 smithi163 ceph-mon[37383]: osdmap e59: 9 total, 9 up, 9 in 2024-08-01T04:11:37.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:11:37.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: osd.8 [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] boot 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: osdmap e58: 9 total, 9 up, 9 in 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:37.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:37.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:37.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:37 smithi160 ceph-mon[37224]: osdmap e59: 9 total, 9 up, 9 in 2024-08-01T04:11:37.899 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:11:37.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_1 2024-08-01T04:11:38.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi163 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi163 to 5696M 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:38 smithi163 ceph-mon[37383]: osdmap e60: 9 total, 9 up, 9 in 2024-08-01T04:11:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi163 2024-08-01T04:11:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi163 to 5696M 2024-08-01T04:11:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:38.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:38.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:38.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:38 smithi003 ceph-mon[32262]: osdmap e60: 9 total, 9 up, 9 in 2024-08-01T04:11:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi163 2024-08-01T04:11:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi163 to 5696M 2024-08-01T04:11:38.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:11:38.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:38.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:11:38.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:11:38.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:38 smithi160 ceph-mon[37224]: osdmap e60: 9 total, 9 up, 9 in 2024-08-01T04:11:38.437 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 -- 172.21.15.3:0/3442269574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04105540 msgr2=0x7f6d0410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 --2- 172.21.15.3:0/3442269574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04105540 0x7f6d0410bdd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009ca0 tx=0x7f6cfc02a8c0 comp rx=0 tx=0).stop 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 -- 172.21.15.3:0/3442269574 shutdown_connections 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 --2- 172.21.15.3:0/3442269574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04105540 0x7f6d0410bdd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 --2- 172.21.15.3:0/3442269574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d04104b80 0x7f6d04105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 --2- 172.21.15.3:0/3442269574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 -- 172.21.15.3:0/3442269574 >> 172.21.15.3:0/3442269574 conn(0x7f6d040ff110 msgr2=0x7f6d04101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.197+0000 7f6d08e00640 1 -- 172.21.15.3:0/3442269574 shutdown_connections 2024-08-01T04:11:39.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.198+0000 7f6d08e00640 1 -- 172.21.15.3:0/3442269574 wait complete. 2024-08-01T04:11:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 Processor -- start 2024-08-01T04:11:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 -- start start 2024-08-01T04:11:39.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04104b80 0x7f6d0419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d04105540 0x7f6d041a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410f650 con 0x7f6d04105540 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410f4d0 con 0x7f6d04103980 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d08e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410f7d0 con 0x7f6d04104b80 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d03400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d03400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:44208/0 (socket says 172.21.15.3:44208) 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6d03400640 1 -- 172.21.15.3:0/2218821183 learned_addr learned my addr 172.21.15.3:0/2218821183 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.199+0000 7f6cfaa00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04104b80 0x7f6d0419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:39.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d04105540 0x7f6d041a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03400640 1 -- 172.21.15.3:0/2218821183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04104b80 msgr2=0x7f6d0419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03400640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04104b80 0x7f6d0419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:39.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03400640 1 -- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d04105540 msgr2=0x7f6d041a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03400640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d04105540 0x7f6d041a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03400640 1 -- 172.21.15.3:0/2218821183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d041a1850 con 0x7f6d04103980 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.200+0000 7f6d03400640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04199c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e9c0 tx=0x7f6cf400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.201+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf400cd80 con 0x7f6d04103980 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.201+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf40035f0 con 0x7f6d04103980 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.201+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4010690 con 0x7f6d04103980 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.201+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d041a1b40 con 0x7f6d04103980 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.201+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041a2030 con 0x7f6d04103980 2024-08-01T04:11:39.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.202+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0410ad30 con 0x7f6d04103980 2024-08-01T04:11:39.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.204+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6cf40037a0 con 0x7f6d04103980 2024-08-01T04:11:39.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.205+0000 7f6d00c00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6ce80a2b80 0x7f6ce80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:11:39.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.205+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5829+0+0 (secure 0 0 0) 0x7f6cf40cea80 con 0x7f6d04103980 2024-08-01T04:11:39.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.206+0000 7f6cfaa00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6ce80a2b80 0x7f6ce80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:11:39.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.206+0000 7f6cfaa00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6ce80a2b80 0x7f6ce80a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009bd0 tx=0x7f6cec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:11:39.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.209+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6cf40d0e20 con 0x7f6d04103980 2024-08-01T04:11:39.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:11:39.416+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6d04062360 con 0x7f6ce80a2b80 2024-08-01T04:11:40.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:40 smithi003 ceph-mon[32262]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:40.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:40 smithi003 ceph-mon[32262]: osdmap e61: 9 total, 9 up, 9 in 2024-08-01T04:11:40.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:40 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:11:40.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:40 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:11:40.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:40 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:40 smithi163 ceph-mon[37383]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:40 smithi163 ceph-mon[37383]: osdmap e61: 9 total, 9 up, 9 in 2024-08-01T04:11:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:40 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:11:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:40 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:11:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:40 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:40 smithi160 ceph-mon[37224]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:40 smithi160 ceph-mon[37224]: osdmap e61: 9 total, 9 up, 9 in 2024-08-01T04:11:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:40 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:11:40.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:40 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:11:40.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:40 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:41.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:41 smithi163 ceph-mon[37383]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:11:41.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:41 smithi163 ceph-mon[37383]: osdmap e62: 9 total, 9 up, 9 in 2024-08-01T04:11:41.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:41 smithi003 ceph-mon[32262]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:11:41.296 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:41 smithi003 ceph-mon[32262]: osdmap e62: 9 total, 9 up, 9 in 2024-08-01T04:11:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:41 smithi160 ceph-mon[37224]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:11:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:41 smithi160 ceph-mon[37224]: osdmap e62: 9 total, 9 up, 9 in 2024-08-01T04:11:42.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:42 smithi163 ceph-mon[37383]: pgmap v198: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-01T04:11:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:42 smithi003 ceph-mon[32262]: pgmap v198: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-01T04:11:42.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:42 smithi160 ceph-mon[37224]: pgmap v198: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-01T04:11:43.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:43 smithi003 ceph-mon[32262]: pgmap v199: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-08-01T04:11:43.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:43 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1647020450' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91a1874-e1c3-40ec-b6de-55b00c8d40e5"}]: dispatch 2024-08-01T04:11:43.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:43 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1647020450' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f91a1874-e1c3-40ec-b6de-55b00c8d40e5"}]': finished 2024-08-01T04:11:43.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:43 smithi003 ceph-mon[32262]: osdmap e63: 10 total, 9 up, 10 in 2024-08-01T04:11:43.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:11:43.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:43 smithi163 ceph-mon[37383]: pgmap v199: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-08-01T04:11:43.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:43 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1647020450' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91a1874-e1c3-40ec-b6de-55b00c8d40e5"}]: dispatch 2024-08-01T04:11:43.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:43 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1647020450' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f91a1874-e1c3-40ec-b6de-55b00c8d40e5"}]': finished 2024-08-01T04:11:43.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:43 smithi163 ceph-mon[37383]: osdmap e63: 10 total, 9 up, 10 in 2024-08-01T04:11:43.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:11:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:43 smithi160 ceph-mon[37224]: pgmap v199: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-08-01T04:11:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:43 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1647020450' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91a1874-e1c3-40ec-b6de-55b00c8d40e5"}]: dispatch 2024-08-01T04:11:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:43 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1647020450' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f91a1874-e1c3-40ec-b6de-55b00c8d40e5"}]': finished 2024-08-01T04:11:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:43 smithi160 ceph-mon[37224]: osdmap e63: 10 total, 9 up, 10 in 2024-08-01T04:11:43.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:11:44.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:44 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3524103183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:11:44.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:44 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3524103183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:11:44.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:44 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3524103183' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:11:45.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:45 smithi160 ceph-mon[37224]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:11:45.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:45 smithi163 ceph-mon[37383]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:11:45.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:45 smithi003 ceph-mon[32262]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:11:47.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:47 smithi003 ceph-mon[32262]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:47.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:47 smithi163 ceph-mon[37383]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:47.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:47 smithi160 ceph-mon[37224]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:49.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:49 smithi003 ceph-mon[32262]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:49.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:49 smithi163 ceph-mon[37383]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:49.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:49 smithi160 ceph-mon[37224]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:51.466 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:51 smithi003 ceph-mon[32262]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:51.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:51 smithi163 ceph-mon[37383]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:51.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:51 smithi160 ceph-mon[37224]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:52.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-01T04:11:52.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:52.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:52 smithi163 ceph-mon[37383]: Deploying daemon osd.9 on smithi003 2024-08-01T04:11:52.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-01T04:11:52.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:52.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:52 smithi003 ceph-mon[32262]: Deploying daemon osd.9 on smithi003 2024-08-01T04:11:52.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-01T04:11:52.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:11:52.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:52 smithi160 ceph-mon[37224]: Deploying daemon osd.9 on smithi003 2024-08-01T04:11:53.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:53 smithi003 ceph-mon[32262]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:53.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:53 smithi163 ceph-mon[37383]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:53 smithi160 ceph-mon[37224]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:55.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:55 smithi003 ceph-mon[32262]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:55.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:55 smithi163 ceph-mon[37383]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:55.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:55 smithi160 ceph-mon[37224]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:57.436 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:57 smithi003 ceph-mon[32262]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:57.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:57 smithi163 ceph-mon[37383]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:57.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:57 smithi160 ceph-mon[37224]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:59.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:11:59 smithi163 ceph-mon[37383]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:59.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:11:59 smithi003 ceph-mon[32262]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:11:59.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:11:59 smithi160 ceph-mon[37224]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:01 smithi003 ceph-mon[32262]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:01.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:01 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:01.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:01 smithi163 ceph-mon[37383]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:01.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:01.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:01.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:01 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:01.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:01 smithi160 ceph-mon[37224]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:01.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:01.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:01.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:01 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:03.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:03 smithi003 ceph-mon[32262]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:03.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:03 smithi163 ceph-mon[37383]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:03.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:03 smithi160 ceph-mon[37224]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:05.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.074+0000 7f6d00c00640 1 -- 172.21.15.3:0/2218821183 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6d04062360 con 0x7f6ce80a2b80 2024-08-01T04:12:05.078 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 9 on host 'smithi003' 2024-08-01T04:12:05.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6ce80a2b80 msgr2=0x7f6ce80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6ce80a2b80 0x7f6ce80a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6cec009bd0 tx=0x7f6cec005f90 comp rx=0 tx=0).stop 2024-08-01T04:12:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 msgr2=0x7f6d04199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:05.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04199c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e9c0 tx=0x7f6cf400ee80 comp rx=0 tx=0).stop 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 shutdown_connections 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6ce80a2b80 0x7f6ce80a5040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d04105540 0x7f6d041a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d04104b80 0x7f6d0419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 --2- 172.21.15.3:0/2218821183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d04103980 0x7f6d04199c30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 >> 172.21.15.3:0/2218821183 conn(0x7f6d040ff110 msgr2=0x7f6d04107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 shutdown_connections 2024-08-01T04:12:05.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:05.078+0000 7f6d08e00640 1 -- 172.21.15.3:0/2218821183 wait complete. 2024-08-01T04:12:05.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:05.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:05.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:05 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:05 smithi163 ceph-mon[37383]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-01T04:12:05.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:05.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:05.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:05 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:05 smithi003 ceph-mon[32262]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-01T04:12:05.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:05.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:05.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:05 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:05.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:05 smithi160 ceph-mon[37224]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-01T04:12:05.913 DEBUG:teuthology.orchestra.run.smithi003:osd.9> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.9.service 2024-08-01T04:12:05.919 INFO:tasks.cephadm:Deploying osd.10 on smithi160 with /dev/vg_nvme/lv_1... 2024-08-01T04:12:05.919 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-01T04:12:06.102 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: osdmap e64: 10 total, 9 up, 10 in 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:06.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:06.184 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:12:06.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: osdmap e64: 10 total, 9 up, 10 in 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:06 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:06.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:06.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-01T04:12:06.305 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: osdmap e64: 10 total, 9 up, 10 in 2024-08-01T04:12:06.306 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-08-01T04:12:06.306 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:06.306 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:06.306 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:06.306 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:06 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:06.800 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:12:06.647+0000 7fc788600640 -1 osd.9 0 waiting for initial osdmap 2024-08-01T04:12:06.800 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:12:06 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:12:06.697+0000 7fc77e600640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:12:07.069 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:07 smithi003 ceph-mon[32262]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:12:07.069 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:07 smithi003 ceph-mon[32262]: osdmap e65: 10 total, 9 up, 10 in 2024-08-01T04:12:07.070 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:07.294 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:07 smithi160 ceph-mon[37224]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:12:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:07 smithi160 ceph-mon[37224]: osdmap e65: 10 total, 9 up, 10 in 2024-08-01T04:12:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:07 smithi160 ceph-mon[37224]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:07.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:07 smithi160 ceph-mon[37224]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' 2024-08-01T04:12:07.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:07.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:07 smithi003 ceph-mon[32262]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:07.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:07 smithi003 ceph-mon[32262]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' 2024-08-01T04:12:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:07 smithi163 ceph-mon[37383]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-08-01T04:12:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:07 smithi163 ceph-mon[37383]: osdmap e65: 10 total, 9 up, 10 in 2024-08-01T04:12:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:07 smithi163 ceph-mon[37383]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-08-01T04:12:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:07 smithi163 ceph-mon[37383]: from='osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975]' entity='osd.9' 2024-08-01T04:12:08.107 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:08 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:12:08.107 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:08 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:12:08.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:08 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:08.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:08 smithi003 ceph-mon[32262]: osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] boot 2024-08-01T04:12:08.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:08 smithi003 ceph-mon[32262]: osdmap e66: 10 total, 10 up, 10 in 2024-08-01T04:12:08.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:08 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:08.300 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:08 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:12:08.300 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:08 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:12:08.300 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:08 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:08.300 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:08 smithi160 ceph-mon[37224]: osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] boot 2024-08-01T04:12:08.301 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:08 smithi160 ceph-mon[37224]: osdmap e66: 10 total, 10 up, 10 in 2024-08-01T04:12:08.301 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:08 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:08.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:08 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:12:08.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:08 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:12:08.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:08 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:08.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:08 smithi163 ceph-mon[37383]: osd.9 [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] boot 2024-08-01T04:12:08.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:08 smithi163 ceph-mon[37383]: osdmap e66: 10 total, 10 up, 10 in 2024-08-01T04:12:08.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:08 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:12:08.673 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-08-01T04:12:08.701 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-08-01T04:12:08.963 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:12:09.373 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:09 smithi160 ceph-mon[37224]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 -- 172.21.15.160:0/1397689409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 msgr2=0x7f0c88101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/1397689409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88101d30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009900 tx=0x7f0c7802a410 comp rx=0 tx=0).stop 2024-08-01T04:12:09.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 -- 172.21.15.160:0/1397689409 shutdown_connections 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/1397689409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c881034f0 0x7f0c88109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/1397689409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 0x7f0c88102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/1397689409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88101d30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 -- 172.21.15.160:0/1397689409 >> 172.21.15.160:0/1397689409 conn(0x7f0c880fd0c0 msgr2=0x7f0c880ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 -- 172.21.15.160:0/1397689409 shutdown_connections 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.453+0000 7f0c8d8da640 1 -- 172.21.15.160:0/1397689409 wait complete. 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 Processor -- start 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 -- start start 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:09.466 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 0x7f0c88198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c881034f0 0x7f0c8819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8810d630 con 0x7f0c881034f0 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8810d4b0 con 0x7f0c88101930 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c8d8da640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8810d7b0 con 0x7f0c88102b30 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c86ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c86ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60446/0 (socket says 172.21.15.160:60446) 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c86ffd640 1 -- 172.21.15.160:0/3088090009 learned_addr learned my addr 172.21.15.160:0/3088090009 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c7e7fc640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 0x7f0c88198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c877fe640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c881034f0 0x7f0c8819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:09.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c86ffd640 1 -- 172.21.15.160:0/3088090009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 msgr2=0x7f0c88198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c86ffd640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 0x7f0c88198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.454+0000 7f0c86ffd640 1 -- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c881034f0 msgr2=0x7f0c8819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c86ffd640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c881034f0 0x7f0c8819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c86ffd640 1 -- 172.21.15.160:0/3088090009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8819f930 con 0x7f0c88101930 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c7e7fc640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 0x7f0c88198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c86ffd640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88197d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c78031610 tx=0x7f0c780043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7803f030 con 0x7f0c88101930 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c78003ce0 con 0x7f0c88101930 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8819fbc0 con 0x7f0c88101930 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78035070 con 0x7f0c88101930 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.455+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c881a00a0 con 0x7f0c88101930 2024-08-01T04:12:09.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.456+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c88108e20 con 0x7f0c88101930 2024-08-01T04:12:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.457+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0c78042030 con 0x7f0c88101930 2024-08-01T04:12:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.457+0000 7f0c8c8d8640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0c740a2b80 0x7f0c740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.458+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6261+0+0 (secure 0 0 0) 0x7f0c780eee90 con 0x7f0c88101930 2024-08-01T04:12:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.458+0000 7f0c7e7fc640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0c740a2b80 0x7f0c740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.459+0000 7f0c7e7fc640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0c740a2b80 0x7f0c740a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009b40 tx=0x7f0c70005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:09.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.462+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0c780bc5f0 con 0x7f0c88101930 2024-08-01T04:12:09.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:09 smithi003 ceph-mon[32262]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:09.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:09 smithi163 ceph-mon[37383]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:09.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:09.638+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0c88062360 con 0x7f0c740a2b80 2024-08-01T04:12:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:10 smithi003 ceph-mon[32262]: osdmap e67: 10 total, 10 up, 10 in 2024-08-01T04:12:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:10 smithi003 ceph-mon[32262]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:12:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:12:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:12:10.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:10.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:10 smithi160 ceph-mon[37224]: osdmap e67: 10 total, 10 up, 10 in 2024-08-01T04:12:10.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:10 smithi160 ceph-mon[37224]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:12:10.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:12:10.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:12:10.385 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:10.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:10 smithi163 ceph-mon[37383]: osdmap e67: 10 total, 10 up, 10 in 2024-08-01T04:12:10.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:10 smithi163 ceph-mon[37383]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:12:10.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:12:10.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:12:10.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi003 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi003 to 4273M 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:11.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:11 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi003 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi003 to 4273M 2024-08-01T04:12:11.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:11.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:11.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:11 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi003 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi003 to 4273M 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:11.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:11.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:11.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:11 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:12.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:12 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/3295344880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]: dispatch 2024-08-01T04:12:12.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:12 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]: dispatch 2024-08-01T04:12:12.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:12 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]': finished 2024-08-01T04:12:12.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:12 smithi160 ceph-mon[37224]: osdmap e68: 11 total, 10 up, 11 in 2024-08-01T04:12:12.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:12 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:12.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:12 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/3295344880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]: dispatch 2024-08-01T04:12:12.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:12 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]: dispatch 2024-08-01T04:12:12.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:12 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]': finished 2024-08-01T04:12:12.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:12 smithi163 ceph-mon[37383]: osdmap e68: 11 total, 10 up, 11 in 2024-08-01T04:12:12.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:12 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:12.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:12 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/3295344880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]: dispatch 2024-08-01T04:12:12.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:12 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]: dispatch 2024-08-01T04:12:12.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:12 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64b32252-f40b-450c-bacc-c37ac7660b3e"}]': finished 2024-08-01T04:12:12.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:12 smithi003 ceph-mon[32262]: osdmap e68: 11 total, 10 up, 11 in 2024-08-01T04:12:12.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:12 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:13 smithi163 ceph-mon[37383]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:13 smithi163 ceph-mon[37383]: from='client.? 172.21.15.160:0/1193851861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:12:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:13 smithi003 ceph-mon[32262]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:13 smithi003 ceph-mon[32262]: from='client.? 172.21.15.160:0/1193851861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:12:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:13 smithi160 ceph-mon[37224]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:13 smithi160 ceph-mon[37224]: from='client.? 172.21.15.160:0/1193851861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:12:15.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:15 smithi163 ceph-mon[37383]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:15.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:15 smithi003 ceph-mon[32262]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:15.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:15 smithi160 ceph-mon[37224]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:17.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:17 smithi163 ceph-mon[37383]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:17 smithi003 ceph-mon[32262]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:17.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:17 smithi160 ceph-mon[37224]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:19.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:19 smithi160 ceph-mon[37224]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:19.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-01T04:12:19.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:19 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:19.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:19 smithi163 ceph-mon[37383]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:19.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-01T04:12:19.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:19 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:19 smithi003 ceph-mon[32262]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-01T04:12:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:19 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:20.531 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:20 smithi160 ceph-mon[37224]: Deploying daemon osd.10 on smithi160 2024-08-01T04:12:20.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:20 smithi163 ceph-mon[37383]: Deploying daemon osd.10 on smithi160 2024-08-01T04:12:20.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:20 smithi003 ceph-mon[32262]: Deploying daemon osd.10 on smithi160 2024-08-01T04:12:21.773 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:21 smithi160 ceph-mon[37224]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:21.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:21 smithi163 ceph-mon[37383]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:21.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:21 smithi003 ceph-mon[32262]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:23.786 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:23 smithi160 ceph-mon[37224]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:23.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:23 smithi163 ceph-mon[37383]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:23.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:23 smithi003 ceph-mon[32262]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:24.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:24 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:24.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:24.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:24.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:24 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:24.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:24.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:24.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:24 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:25.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:25 smithi163 ceph-mon[37383]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:25.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:25 smithi003 ceph-mon[32262]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:25.852 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:25 smithi160 ceph-mon[37224]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:27 smithi163 ceph-mon[37383]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:27 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:27 smithi003 ceph-mon[32262]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:27 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.842 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:27 smithi160 ceph-mon[37224]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:27.842 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.842 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.842 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:27.842 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:27.842 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:27 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:27.900 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.898+0000 7f0c8c8d8640 1 -- 172.21.15.160:0/3088090009 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0c88062360 con 0x7f0c740a2b80 2024-08-01T04:12:27.901 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 10 on host 'smithi160' 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0c740a2b80 msgr2=0x7f0c740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0c740a2b80 0x7f0c740a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009b40 tx=0x7f0c70005e50 comp rx=0 tx=0).stop 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 msgr2=0x7f0c88197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88197d10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c78031610 tx=0x7f0c780043b0 comp rx=0 tx=0).stop 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 shutdown_connections 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0c740a2b80 0x7f0c740a5040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c881034f0 0x7f0c8819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c88102b30 0x7f0c88198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:27.903 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 --2- 172.21.15.160:0/3088090009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c88101930 0x7f0c88197d10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:27.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 >> 172.21.15.160:0/3088090009 conn(0x7f0c880fd0c0 msgr2=0x7f0c88105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:27.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 shutdown_connections 2024-08-01T04:12:27.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-08-01T04:12:27.900+0000 7f0c8d8da640 1 -- 172.21.15.160:0/3088090009 wait complete. 2024-08-01T04:12:28.508 DEBUG:teuthology.orchestra.run.smithi160:osd.10> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.10.service 2024-08-01T04:12:28.511 INFO:tasks.cephadm:Deploying osd.11 on smithi163 with /dev/vg_nvme/lv_1... 2024-08-01T04:12:28.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-01T04:12:28.774 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:12:28.791 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:28 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:28.791 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:28 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:28.792 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:28 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:28.792 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:28 smithi160 ceph-mon[37224]: from='osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-01T04:12:28.792 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:28 smithi160 ceph-mon[37224]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-01T04:12:28.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:28 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:28.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:28 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:28.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:28 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:28.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:28 smithi163 ceph-mon[37383]: from='osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-01T04:12:28.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:28 smithi163 ceph-mon[37383]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-01T04:12:28.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:28 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:28.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:28 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:28.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:28 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:28.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:28 smithi003 ceph-mon[32262]: from='osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-01T04:12:28.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:28 smithi003 ceph-mon[32262]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-01T04:12:30.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:29 smithi160 ceph-mon[37224]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:30.179 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:29 smithi160 ceph-mon[37224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-01T04:12:30.180 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:29 smithi160 ceph-mon[37224]: from='osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:12:30.180 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:29 smithi160 ceph-mon[37224]: osdmap e69: 11 total, 10 up, 11 in 2024-08-01T04:12:30.180 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:30.180 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:29 smithi160 ceph-mon[37224]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:12:30.225 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:29 smithi163 ceph-mon[37383]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:30.225 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:29 smithi163 ceph-mon[37383]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-01T04:12:30.225 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:29 smithi163 ceph-mon[37383]: from='osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:12:30.225 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:29 smithi163 ceph-mon[37383]: osdmap e69: 11 total, 10 up, 11 in 2024-08-01T04:12:30.225 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:30.225 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:29 smithi163 ceph-mon[37383]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:12:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:29 smithi003 ceph-mon[32262]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:29 smithi003 ceph-mon[32262]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-01T04:12:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:29 smithi003 ceph-mon[32262]: from='osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:12:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:29 smithi003 ceph-mon[32262]: osdmap e69: 11 total, 10 up, 11 in 2024-08-01T04:12:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:29 smithi003 ceph-mon[32262]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-08-01T04:12:30.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:12:30.334+0000 7f77795df640 -1 osd.10 0 waiting for initial osdmap 2024-08-01T04:12:30.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:12:30.348+0000 7f7770beb640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:12:31.007 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-mon[37224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:12:31.007 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-mon[37224]: osdmap e70: 11 total, 10 up, 11 in 2024-08-01T04:12:31.007 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:31.007 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:31.007 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:30 smithi160 ceph-mon[37224]: from='osd.10 ' entity='osd.10' 2024-08-01T04:12:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:30 smithi163 ceph-mon[37383]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:12:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:30 smithi163 ceph-mon[37383]: osdmap e70: 11 total, 10 up, 11 in 2024-08-01T04:12:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:30 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:30 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:30 smithi163 ceph-mon[37383]: from='osd.10 ' entity='osd.10' 2024-08-01T04:12:31.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:30 smithi003 ceph-mon[32262]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-08-01T04:12:31.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:30 smithi003 ceph-mon[32262]: osdmap e70: 11 total, 10 up, 11 in 2024-08-01T04:12:31.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:30 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:31.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:30 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:31.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:30 smithi003 ceph-mon[32262]: from='osd.10 ' entity='osd.10' 2024-08-01T04:12:31.303 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-08-01T04:12:31.329 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch daemon add osd smithi163:vg_nvme/lv_1 2024-08-01T04:12:31.595 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:12:32.069 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 -- 172.21.15.163:0/590278661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 msgr2=0x7ff5e8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/590278661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8104800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009900 tx=0x7ff5e402a410 comp rx=0 tx=0).stop 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 -- 172.21.15.163:0/590278661 shutdown_connections 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/590278661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e8107670 0x7ff5e8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/590278661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5e8104d40 0x7ff5e8107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/590278661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8104800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 -- 172.21.15.163:0/590278661 >> 172.21.15.163:0/590278661 conn(0x7ff5e80fbfb0 msgr2=0x7ff5e80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 -- 172.21.15.163:0/590278661 shutdown_connections 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 -- 172.21.15.163:0/590278661 wait complete. 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 Processor -- start 2024-08-01T04:12:32.070 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.046+0000 7ff5efd0c640 1 -- start start 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5efd0c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5efd0c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5e8104d40 0x7ff5e81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5efd0c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e8107670 0x7ff5e819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5efd0c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e810d650 con 0x7ff5e8104d40 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5efd0c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e810d4d0 con 0x7ff5e8107670 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5efd0c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e810d7d0 con 0x7ff5e8102410 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37124/0 (socket says 172.21.15.163:37124) 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 -- 172.21.15.163:0/2735259743 learned_addr learned my addr 172.21.15.163:0/2735259743 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5ee2a9640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e8107670 0x7ff5e819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 -- 172.21.15.163:0/2735259743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e8107670 msgr2=0x7ff5e819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:32.071 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e8107670 0x7ff5e819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 -- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5e8104d40 msgr2=0x7ff5e81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5e8104d40 0x7ff5e81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 -- 172.21.15.163:0/2735259743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e819f880 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.047+0000 7ff5edaa8640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8197c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5e402aa00 tx=0x7ff5e4032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.048+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e403f030 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.048+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e819fb10 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.048+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e4003ce0 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.048+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4035070 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.048+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e81a0040 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.049+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e80fdaa0 con 0x7ff5e8102410 2024-08-01T04:12:32.072 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.050+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff5e4042030 con 0x7ff5e8102410 2024-08-01T04:12:32.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.050+0000 7ff5deffd640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5d40a2b80 0x7ff5d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:32.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.050+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6723+0+0 (secure 0 0 0) 0x7ff5e40ef130 con 0x7ff5e8102410 2024-08-01T04:12:32.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.051+0000 7ff5ed2a7640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5d40a2b80 0x7ff5d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:32.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.052+0000 7ff5ed2a7640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5d40a2b80 0x7ff5d40a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806a780 tx=0x7ff5d8006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:32.073 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.054+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff5e40bc6c0 con 0x7ff5e8102410 2024-08-01T04:12:32.252 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:32.251+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff5e8062360 con 0x7ff5d40a2b80 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] boot 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: osdmap e71: 11 total, 11 up, 11 in 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.253 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:32.254 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] boot 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: osdmap e71: 11 total, 11 up, 11 in 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:32.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: osd.10 [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] boot 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: osdmap e71: 11 total, 11 up, 11 in 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:32.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:32.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:32.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:32.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:32 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi160 2024-08-01T04:12:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:32 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi160 to 4272M 2024-08-01T04:12:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:32 smithi163 ceph-mon[37383]: osdmap e72: 11 total, 11 up, 11 in 2024-08-01T04:12:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:12:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:12:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:32 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:33.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:32 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi160 2024-08-01T04:12:33.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:32 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi160 to 4272M 2024-08-01T04:12:33.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:32 smithi003 ceph-mon[32262]: osdmap e72: 11 total, 11 up, 11 in 2024-08-01T04:12:33.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:12:33.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:12:33.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:32 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:32 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi160 2024-08-01T04:12:33.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:32 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi160 to 4272M 2024-08-01T04:12:33.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:32 smithi160 ceph-mon[37224]: osdmap e72: 11 total, 11 up, 11 in 2024-08-01T04:12:33.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-01T04:12:33.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-01T04:12:33.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:32 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:34.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:33 smithi163 ceph-mon[37383]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:33 smithi163 ceph-mon[37383]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:12:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:33 smithi163 ceph-mon[37383]: osdmap e73: 11 total, 11 up, 11 in 2024-08-01T04:12:34.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:33 smithi003 ceph-mon[32262]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:34.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:33 smithi003 ceph-mon[32262]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:12:34.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:33 smithi003 ceph-mon[32262]: osdmap e73: 11 total, 11 up, 11 in 2024-08-01T04:12:34.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:33 smithi160 ceph-mon[37224]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:34.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:33 smithi160 ceph-mon[37224]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi163:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:12:34.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:33 smithi160 ceph-mon[37224]: osdmap e73: 11 total, 11 up, 11 in 2024-08-01T04:12:35.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:34 smithi003 ceph-mon[32262]: osdmap e74: 11 total, 11 up, 11 in 2024-08-01T04:12:35.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:34 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/433966010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]: dispatch 2024-08-01T04:12:35.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:34 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]: dispatch 2024-08-01T04:12:35.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:34 smithi003 ceph-mon[32262]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]': finished 2024-08-01T04:12:35.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:34 smithi003 ceph-mon[32262]: osdmap e75: 12 total, 11 up, 12 in 2024-08-01T04:12:35.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:34 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:35.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:34 smithi163 ceph-mon[37383]: osdmap e74: 11 total, 11 up, 11 in 2024-08-01T04:12:35.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:34 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/433966010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]: dispatch 2024-08-01T04:12:35.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:34 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]: dispatch 2024-08-01T04:12:35.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:34 smithi163 ceph-mon[37383]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]': finished 2024-08-01T04:12:35.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:34 smithi163 ceph-mon[37383]: osdmap e75: 12 total, 11 up, 12 in 2024-08-01T04:12:35.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:34 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:34 smithi160 ceph-mon[37224]: osdmap e74: 11 total, 11 up, 11 in 2024-08-01T04:12:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:34 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/433966010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]: dispatch 2024-08-01T04:12:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:34 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]: dispatch 2024-08-01T04:12:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:34 smithi160 ceph-mon[37224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2"}]': finished 2024-08-01T04:12:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:34 smithi160 ceph-mon[37224]: osdmap e75: 12 total, 11 up, 12 in 2024-08-01T04:12:35.304 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:34 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:36.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:35 smithi163 ceph-mon[37383]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-01T04:12:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:35 smithi163 ceph-mon[37383]: from='client.? 172.21.15.163:0/2005885962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:12:36.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:35 smithi003 ceph-mon[32262]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-01T04:12:36.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:35 smithi003 ceph-mon[32262]: from='client.? 172.21.15.163:0/2005885962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:12:36.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:35 smithi160 ceph-mon[37224]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-01T04:12:36.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:35 smithi160 ceph-mon[37224]: from='client.? 172.21.15.163:0/2005885962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-01T04:12:38.267 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:37 smithi163 ceph-mon[37383]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-08-01T04:12:38.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:37 smithi003 ceph-mon[32262]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-08-01T04:12:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:37 smithi160 ceph-mon[37224]: pgmap v239: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-08-01T04:12:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:39 smithi163 ceph-mon[37383]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:12:40.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:39 smithi003 ceph-mon[32262]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:12:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:39 smithi160 ceph-mon[37224]: pgmap v240: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-01T04:12:42.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:41 smithi163 ceph-mon[37383]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-08-01T04:12:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-01T04:12:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:41 smithi003 ceph-mon[32262]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-08-01T04:12:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-01T04:12:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:42.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:41 smithi160 ceph-mon[37224]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-08-01T04:12:42.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-01T04:12:42.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:43.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:42 smithi163 ceph-mon[37383]: Deploying daemon osd.11 on smithi163 2024-08-01T04:12:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:42 smithi003 ceph-mon[32262]: Deploying daemon osd.11 on smithi163 2024-08-01T04:12:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:42 smithi160 ceph-mon[37224]: Deploying daemon osd.11 on smithi163 2024-08-01T04:12:44.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:43 smithi163 ceph-mon[37383]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:44.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:43 smithi003 ceph-mon[32262]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:44.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:43 smithi160 ceph-mon[37224]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:46.140 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:46 smithi163 ceph-mon[37383]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:46.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:46 smithi003 ceph-mon[32262]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:46.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:46 smithi160 ceph-mon[37224]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:48.187 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:48 smithi163 ceph-mon[37383]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:48.188 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:48.188 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:48.188 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:48.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:48 smithi003 ceph-mon[32262]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:48.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:48 smithi160 ceph-mon[37224]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:48.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:48.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:48.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:50.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:50 smithi163 ceph-mon[37383]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:50.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:50 smithi003 ceph-mon[32262]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:50.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:50 smithi160 ceph-mon[37224]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:51.360 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.357+0000 7ff5deffd640 1 -- 172.21.15.163:0/2735259743 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff5e8062360 con 0x7ff5d40a2b80 2024-08-01T04:12:51.360 INFO:teuthology.orchestra.run.smithi163.stdout:Created osd(s) 11 on host 'smithi163' 2024-08-01T04:12:51.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5d40a2b80 msgr2=0x7ff5d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:51.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5d40a2b80 0x7ff5d40a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806a780 tx=0x7ff5d8006c90 comp rx=0 tx=0).stop 2024-08-01T04:12:51.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 msgr2=0x7ff5e8197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:51.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8197c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5e402aa00 tx=0x7ff5e4032da0 comp rx=0 tx=0).stop 2024-08-01T04:12:51.362 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 shutdown_connections 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff5d40a2b80 0x7ff5d40a5040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5e8107670 0x7ff5e819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5e8104d40 0x7ff5e81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 --2- 172.21.15.163:0/2735259743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e8102410 0x7ff5e8197c60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 >> 172.21.15.163:0/2735259743 conn(0x7ff5e80fbfb0 msgr2=0x7ff5e8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 shutdown_connections 2024-08-01T04:12:51.363 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:12:51.360+0000 7ff5efd0c640 1 -- 172.21.15.163:0/2735259743 wait complete. 2024-08-01T04:12:52.094 DEBUG:teuthology.orchestra.run.smithi163:osd.11> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.11.service 2024-08-01T04:12:52.097 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-01T04:12:52.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd stat -f json 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: from='osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:52 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:52.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: from='osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-01T04:12:52.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-01T04:12:52.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:52 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:52.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: from='osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-01T04:12:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-01T04:12:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:52 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:12:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.342+0000 7f4b47e00640 1 -- 172.21.15.3:0/2515145915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 msgr2=0x7f4b48102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.342+0000 7f4b47e00640 1 --2- 172.21.15.3:0/2515145915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48102df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009ca0 tx=0x7f4b3802a8b0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 -- 172.21.15.3:0/2515145915 shutdown_connections 2024-08-01T04:12:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 --2- 172.21.15.3:0/2515145915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 0x7f4b48109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 --2- 172.21.15.3:0/2515145915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48102df0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 --2- 172.21.15.3:0/2515145915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b48101770 0x7f4b48101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 -- 172.21.15.3:0/2515145915 >> 172.21.15.3:0/2515145915 conn(0x7f4b480fcf20 msgr2=0x7f4b480ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 -- 172.21.15.3:0/2515145915 shutdown_connections 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.343+0000 7f4b47e00640 1 -- 172.21.15.3:0/2515145915 wait complete. 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.344+0000 7f4b47e00640 1 Processor -- start 2024-08-01T04:12:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.344+0000 7f4b47e00640 1 -- start start 2024-08-01T04:12:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b48101770 0x7f4b48197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 0x7f4b4819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d420 con 0x7f4b48102970 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d2a0 con 0x7f4b48101770 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d5a0 con 0x7f4b48103330 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b46000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 0x7f4b4819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b46000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45662/0 (socket says 172.21.15.3:45662) 2024-08-01T04:12:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b46000640 1 -- 172.21.15.3:0/294241393 learned_addr learned my addr 172.21.15.3:0/294241393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b46a00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b48101770 0x7f4b48197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.345+0000 7f4b47400640 1 -- 172.21.15.3:0/294241393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b48101770 msgr2=0x7f4b48197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b47400640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b48101770 0x7f4b48197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b47400640 1 -- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 msgr2=0x7f4b48197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b47400640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b47400640 1 -- 172.21.15.3:0/294241393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4819f640 con 0x7f4b48103330 2024-08-01T04:12:53.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b47400640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 0x7f4b4819efd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e970 tx=0x7f4b4000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:53.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000ccc0 con 0x7f4b48103330 2024-08-01T04:12:53.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b40005020 con 0x7f4b48103330 2024-08-01T04:12:53.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4819f930 con 0x7f4b48103330 2024-08-01T04:12:53.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.346+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40004a00 con 0x7f4b48103330 2024-08-01T04:12:53.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.347+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4819fe70 con 0x7f4b48103330 2024-08-01T04:12:53.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.348+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b48108b20 con 0x7f4b48103330 2024-08-01T04:12:53.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.350+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4b400051d0 con 0x7f4b48103330 2024-08-01T04:12:53.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.351+0000 7f4b3f400640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b2c0a2c30 0x7f4b2c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:53.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.351+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6831+0+0 (secure 0 0 0) 0x7f4b400cea20 con 0x7f4b48103330 2024-08-01T04:12:53.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.351+0000 7f4b46a00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b2c0a2c30 0x7f4b2c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.352+0000 7f4b46a00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b2c0a2c30 0x7f4b2c0a50f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4b300095f0 tx=0x7f4b30005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.355+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4b4009beb0 con 0x7f4b48103330 2024-08-01T04:12:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.555+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4b4802d5e0 con 0x7f4b48103330 2024-08-01T04:12:53.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.556+0000 7f4b3f400640 1 -- 172.21.15.3:0/294241393 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f4b4009b7b0 con 0x7f4b48103330 2024-08-01T04:12:53.558 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:12:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.559+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b2c0a2c30 msgr2=0x7f4b2c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.559+0000 7f4b47e00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b2c0a2c30 0x7f4b2c0a50f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4b300095f0 tx=0x7f4b30005f90 comp rx=0 tx=0).stop 2024-08-01T04:12:53.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.559+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 msgr2=0x7f4b4819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.559+0000 7f4b47e00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 0x7f4b4819efd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000e970 tx=0x7f4b4000ee30 comp rx=0 tx=0).stop 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 shutdown_connections 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b2c0a2c30 0x7f4b2c0a50f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b48103330 0x7f4b4819efd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b48102970 0x7f4b48197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 --2- 172.21.15.3:0/294241393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b48101770 0x7f4b48197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:53.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 >> 172.21.15.3:0/294241393 conn(0x7f4b480fcf20 msgr2=0x7f4b48105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 shutdown_connections 2024-08-01T04:12:53.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:53.560+0000 7f4b47e00640 1 -- 172.21.15.3:0/294241393 wait complete. 2024-08-01T04:12:53.615 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-mon[37383]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:53.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-mon[37383]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-01T04:12:53.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-mon[37383]: from='osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:12:53.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-mon[37383]: osdmap e76: 12 total, 11 up, 12 in 2024-08-01T04:12:53.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:53.616 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-mon[37383]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:12:53.616 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:12:53.363+0000 7fe7e3d5a640 -1 osd.11 0 waiting for initial osdmap 2024-08-01T04:12:53.616 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:12:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:12:53.381+0000 7fe7dfb82640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-01T04:12:53.630 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:53 smithi003 ceph-mon[32262]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:53.630 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:53 smithi003 ceph-mon[32262]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-01T04:12:53.630 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:53 smithi003 ceph-mon[32262]: from='osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:12:53.630 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:53 smithi003 ceph-mon[32262]: osdmap e76: 12 total, 11 up, 12 in 2024-08-01T04:12:53.630 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:53.630 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:53 smithi003 ceph-mon[32262]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:12:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:53 smithi160 ceph-mon[37224]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:53 smithi160 ceph-mon[37224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-01T04:12:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:53 smithi160 ceph-mon[37224]: from='osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:12:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:53 smithi160 ceph-mon[37224]: osdmap e76: 12 total, 11 up, 12 in 2024-08-01T04:12:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:53.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:53 smithi160 ceph-mon[37224]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-08-01T04:12:54.605 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1722485550,"num_in_osds":12,"osd_in_since":1722485554,"num_remapped_pgs":0} 2024-08-01T04:12:54.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:54 smithi160 ceph-mon[37224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:12:54.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:54 smithi160 ceph-mon[37224]: osdmap e77: 12 total, 11 up, 12 in 2024-08-01T04:12:54.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:54 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:54.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:54 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:54.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:54 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/294241393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-01T04:12:54.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:54 smithi163 ceph-mon[37383]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:12:54.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:54 smithi163 ceph-mon[37383]: osdmap e77: 12 total, 11 up, 12 in 2024-08-01T04:12:54.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:54 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:54.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:54 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:54.664 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:54 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/294241393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-01T04:12:54.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:54 smithi003 ceph-mon[32262]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-08-01T04:12:54.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:54 smithi003 ceph-mon[32262]: osdmap e77: 12 total, 11 up, 12 in 2024-08-01T04:12:54.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:54 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:54.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:54 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:54.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:54 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/294241393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-01T04:12:55.606 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd stat -f json 2024-08-01T04:12:55.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: purged_snaps scrub starts 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: purged_snaps scrub ok 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] boot 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: osdmap e78: 12 total, 12 up, 12 in 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: Detected new or changed devices on smithi163 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:55.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:55 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.688 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: purged_snaps scrub starts 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: purged_snaps scrub ok 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] boot 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: osdmap e78: 12 total, 12 up, 12 in 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: Detected new or changed devices on smithi163 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:55.690 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:55 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: purged_snaps scrub starts 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: purged_snaps scrub ok 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: osd.11 [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] boot 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: osdmap e78: 12 total, 12 up, 12 in 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: Detected new or changed devices on smithi163 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:12:55.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:55 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:12:56.103 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:12:56.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:56 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi163 to 4272M 2024-08-01T04:12:56.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:56 smithi003 ceph-mon[32262]: osdmap e79: 12 total, 12 up, 12 in 2024-08-01T04:12:56.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:56 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi163 to 4272M 2024-08-01T04:12:56.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:56 smithi160 ceph-mon[37224]: osdmap e79: 12 total, 12 up, 12 in 2024-08-01T04:12:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:56 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi163 to 4272M 2024-08-01T04:12:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:56 smithi163 ceph-mon[37383]: osdmap e79: 12 total, 12 up, 12 in 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 -- 172.21.15.3:0/1938918519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc102960 msgr2=0x7f54fc102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 --2- 172.21.15.3:0/1938918519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc102960 0x7f54fc102de0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009900 tx=0x7f54e802a3f0 comp rx=0 tx=0).stop 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 -- 172.21.15.3:0/1938918519 shutdown_connections 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 --2- 172.21.15.3:0/1938918519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc103320 0x7f54fc109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 --2- 172.21.15.3:0/1938918519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc102960 0x7f54fc102de0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 --2- 172.21.15.3:0/1938918519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54fc101760 0x7f54fc101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.873+0000 7f5501400640 1 -- 172.21.15.3:0/1938918519 >> 172.21.15.3:0/1938918519 conn(0x7f54fc0fcef0 msgr2=0x7f54fc0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.874+0000 7f5501400640 1 -- 172.21.15.3:0/1938918519 shutdown_connections 2024-08-01T04:12:56.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.874+0000 7f5501400640 1 -- 172.21.15.3:0/1938918519 wait complete. 2024-08-01T04:12:56.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 Processor -- start 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 -- start start 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc101760 0x7f54fc075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 0x7f54fc074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54fc103320 0x7f54fc074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10d410 con 0x7f54fc101760 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10d290 con 0x7f54fc103320 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.875+0000 7f5501400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc10d590 con 0x7f54fc102960 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 0x7f54fc074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc101760 0x7f54fc075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:56.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 0x7f54fc074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50670/0 (socket says 172.21.15.3:50670) 2024-08-01T04:12:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 -- 172.21.15.3:0/1557526343 learned_addr learned my addr 172.21.15.3:0/1557526343 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:12:56.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f5500a00640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54fc103320 0x7f54fc074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 -- 172.21.15.3:0/1557526343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54fc103320 msgr2=0x7f54fc074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54fc103320 0x7f54fc074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 -- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc101760 msgr2=0x7f54fc075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc101760 0x7f54fc075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.876+0000 7f54fb400640 1 -- 172.21.15.3:0/1557526343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc074d90 con 0x7f54fc102960 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.877+0000 7f54fb400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 0x7f54fc074310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54e8031440 tx=0x7f54e8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.877+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e803f030 con 0x7f54fc102960 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.877+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e8003ce0 con 0x7f54fc102960 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.877+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc075020 con 0x7f54fc102960 2024-08-01T04:12:56.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.877+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8035070 con 0x7f54fc102960 2024-08-01T04:12:56.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.877+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc1a6470 con 0x7f54fc102960 2024-08-01T04:12:56.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.878+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54fc101b60 con 0x7f54fc102960 2024-08-01T04:12:56.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.880+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f54e8042030 con 0x7f54fc102960 2024-08-01T04:12:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.882+0000 7f54f8c00640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f54e00a2b80 0x7f54e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.882+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f54e80ef0c0 con 0x7f54fc102960 2024-08-01T04:12:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.882+0000 7f54fbe00640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f54e00a2b80 0x7f54e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:56.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.883+0000 7f54fbe00640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f54e00a2b80 0x7f54e00a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54fc1027c0 tx=0x7f54ec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:56.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:56.886+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f54e80bc420 con 0x7f54fc102960 2024-08-01T04:12:57.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.084+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f54fc108550 con 0x7f54fc102960 2024-08-01T04:12:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.085+0000 7f54f8c00640 1 -- 172.21.15.3:0/1557526343 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f54e80bbd20 con 0x7f54fc102960 2024-08-01T04:12:57.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:12:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f54e00a2b80 msgr2=0x7f54e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f54e00a2b80 0x7f54e00a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54fc1027c0 tx=0x7f54ec005f90 comp rx=0 tx=0).stop 2024-08-01T04:12:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 msgr2=0x7f54fc074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 0x7f54fc074310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f54e8031440 tx=0x7f54e8032ce0 comp rx=0 tx=0).stop 2024-08-01T04:12:57.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 shutdown_connections 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f54e00a2b80 0x7f54e00a5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54fc103320 0x7f54fc074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54fc102960 0x7f54fc074310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 --2- 172.21.15.3:0/1557526343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f54fc101760 0x7f54fc075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 >> 172.21.15.3:0/1557526343 conn(0x7f54fc0fcef0 msgr2=0x7f54fc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 shutdown_connections 2024-08-01T04:12:57.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:57.088+0000 7f5501400640 1 -- 172.21.15.3:0/1557526343 wait complete. 2024-08-01T04:12:57.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:57 smithi160 ceph-mon[37224]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:12:57.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:57 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1557526343' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-01T04:12:57.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:57 smithi163 ceph-mon[37383]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:12:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:57 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1557526343' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-01T04:12:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:57 smithi003 ceph-mon[32262]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:12:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:57 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1557526343' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-01T04:12:58.128 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1722485574,"num_in_osds":12,"osd_in_since":1722485554,"num_remapped_pgs":0} 2024-08-01T04:12:58.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:12:58.610 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:12:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 -- 172.21.15.3:0/1132655514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c0100450 msgr2=0x7fa6c0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 --2- 172.21.15.3:0/1132655514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c0100450 0x7fa6c0100830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009860 tx=0x7fa6b002a340 comp rx=0 tx=0).stop 2024-08-01T04:12:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 -- 172.21.15.3:0/1132655514 shutdown_connections 2024-08-01T04:12:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 --2- 172.21.15.3:0/1132655514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c010e000 0x7fa6c01103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 --2- 172.21.15.3:0/1132655514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 --2- 172.21.15.3:0/1132655514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c0100450 0x7fa6c0100830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 -- 172.21.15.3:0/1132655514 >> 172.21.15.3:0/1132655514 conn(0x7fa6c00fc340 msgr2=0x7fa6c00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 -- 172.21.15.3:0/1132655514 shutdown_connections 2024-08-01T04:12:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.358+0000 7fa6c5600640 1 -- 172.21.15.3:0/1132655514 wait complete. 2024-08-01T04:12:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 Processor -- start 2024-08-01T04:12:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 -- start start 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c0100450 0x7fa6c019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c010e000 0x7fa6c019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0113d00 con 0x7fa6c0100e00 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0113b80 con 0x7fa6c010e000 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.360+0000 7fa6c5600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0113e80 con 0x7fa6c0100450 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bf400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45706/0 (socket says 172.21.15.3:45706) 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bf400640 1 -- 172.21.15.3:0/553774059 learned_addr learned my addr 172.21.15.3:0/553774059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bfe00640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c0100450 0x7fa6c019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6c4c00640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c010e000 0x7fa6c019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bfe00640 1 -- 172.21.15.3:0/553774059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c010e000 msgr2=0x7fa6c019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bfe00640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c010e000 0x7fa6c019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bfe00640 1 -- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 msgr2=0x7fa6c019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bfe00640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bfe00640 1 -- 172.21.15.3:0/553774059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c019f210 con 0x7fa6c0100450 2024-08-01T04:12:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.361+0000 7fa6bf400640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:12:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.362+0000 7fa6bfe00640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c0100450 0x7fa6c019a260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6b002a860 tx=0x7fa6b0033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.362+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0037040 con 0x7fa6c0100450 2024-08-01T04:12:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.362+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b0033d50 con 0x7fa6c0100450 2024-08-01T04:12:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.362+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c019f4a0 con 0x7fa6c0100450 2024-08-01T04:12:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.362+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0003960 con 0x7fa6c0100450 2024-08-01T04:12:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.362+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01a6de0 con 0x7fa6c0100450 2024-08-01T04:12:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.363+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c006a580 con 0x7fa6c0100450 2024-08-01T04:12:59.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.366+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa6b0042020 con 0x7fa6c0100450 2024-08-01T04:12:59.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.367+0000 7fa6bcc00640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa6a40a2b80 0x7fa6a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:12:59.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.367+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa6b00ee710 con 0x7fa6c0100450 2024-08-01T04:12:59.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.367+0000 7fa6bf400640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa6a40a2b80 0x7fa6a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:12:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.368+0000 7fa6bf400640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa6a40a2b80 0x7fa6a40a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6c010fe70 tx=0x7fa6ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:12:59.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.369+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa6b0049020 con 0x7fa6c0100450 2024-08-01T04:12:59.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:12:59 smithi003 ceph-mon[32262]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:12:59.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.566+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6c0109190 con 0x7fa6c0100450 2024-08-01T04:12:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.568+0000 7fa6bcc00640 1 -- 172.21.15.3:0/553774059 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18423 (secure 0 0 0) 0x7fa6b00bba70 con 0x7fa6c0100450 2024-08-01T04:12:59.570 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:12:59.570 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:12:55.380843+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:12:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa6a40a2b80 msgr2=0x7fa6a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:59.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa6a40a2b80 0x7fa6a40a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa6c010fe70 tx=0x7fa6ac009260 comp rx=0 tx=0).stop 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c0100450 msgr2=0x7fa6c019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c0100450 0x7fa6c019a260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6b002a860 tx=0x7fa6b0033b60 comp rx=0 tx=0).stop 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 shutdown_connections 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa6a40a2b80 0x7fa6a40a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6c010e000 0x7fa6c019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6c0100e00 0x7fa6c019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.571+0000 7fa6c5600640 1 --2- 172.21.15.3:0/553774059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c0100450 0x7fa6c019a260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:12:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.572+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 >> 172.21.15.3:0/553774059 conn(0x7fa6c00fc340 msgr2=0x7fa6c010f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:12:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.572+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 shutdown_connections 2024-08-01T04:12:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:12:59.572+0000 7fa6c5600640 1 -- 172.21.15.3:0/553774059 wait complete. 2024-08-01T04:12:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:12:59 smithi163 ceph-mon[37383]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:12:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:12:59 smithi160 ceph-mon[37224]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:00.320 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-01T04:09:04.723203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-01T04:13:00.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-01T04:13:00.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:00 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/553774059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:13:00.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:00 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/553774059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:13:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:00.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:00 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/553774059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 -- 172.21.15.3:0/4281194865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4112760 msgr2=0x7f5bb4114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 --2- 172.21.15.3:0/4281194865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4112760 0x7f5bb4114b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009ca0 tx=0x7f5bac02a8c0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 -- 172.21.15.3:0/4281194865 shutdown_connections 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 --2- 172.21.15.3:0/4281194865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4112760 0x7f5bb4114b50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 --2- 172.21.15.3:0/4281194865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4076d70 0x7f5bb4112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 --2- 172.21.15.3:0/4281194865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bb4076450 0x7f5bb4076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.479+0000 7f5bbb200640 1 -- 172.21.15.3:0/4281194865 >> 172.21.15.3:0/4281194865 conn(0x7f5bb40ff7c0 msgr2=0x7f5bb4101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.480+0000 7f5bbb200640 1 -- 172.21.15.3:0/4281194865 shutdown_connections 2024-08-01T04:13:01.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.480+0000 7f5bbb200640 1 -- 172.21.15.3:0/4281194865 wait complete. 2024-08-01T04:13:01.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 Processor -- start 2024-08-01T04:13:01.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 -- start start 2024-08-01T04:13:01.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bb4076450 0x7f5bb419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:01.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 0x7f5bb419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:01.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4112760 0x7f5bb41a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:01.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41182f0 con 0x7f5bb4076d70 2024-08-01T04:13:01.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4118170 con 0x7f5bb4076450 2024-08-01T04:13:01.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.481+0000 7f5bbb200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4118470 con 0x7f5bb4112760 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bab400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 0x7f5bb419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bab400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 0x7f5bb419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45736/0 (socket says 172.21.15.3:45736) 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bab400640 1 -- 172.21.15.3:0/2523669695 learned_addr learned my addr 172.21.15.3:0/2523669695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb9200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4112760 0x7f5bb41a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb3e00640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bb4076450 0x7f5bb419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb9200640 1 -- 172.21.15.3:0/2523669695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bb4076450 msgr2=0x7f5bb419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:01.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb9200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bb4076450 0x7f5bb419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb9200640 1 -- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 msgr2=0x7f5bb419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb9200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 0x7f5bb419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bb9200640 1 -- 172.21.15.3:0/2523669695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb41a36e0 con 0x7f5bb4112760 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.482+0000 7f5bab400640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 0x7f5bb419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.483+0000 7f5bb9200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4112760 0x7f5bb41a3000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5bac02aeb0 tx=0x7f5bac003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.483+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac031650 con 0x7f5bb4112760 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.483+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bac0317f0 con 0x7f5bb4112760 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.483+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb41a3970 con 0x7f5bb4112760 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.483+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac02f990 con 0x7f5bb4112760 2024-08-01T04:13:01.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.483+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb41ab2b0 con 0x7f5bb4112760 2024-08-01T04:13:01.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.484+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb4076830 con 0x7f5bb4112760 2024-08-01T04:13:01.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.486+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5bac02fb30 con 0x7f5bb4112760 2024-08-01T04:13:01.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.488+0000 7f5bb1600640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5b980a2b80 0x7f5b980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:01.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.488+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f5bac0efe60 con 0x7f5bb4112760 2024-08-01T04:13:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.488+0000 7f5bb3e00640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5b980a2b80 0x7f5b980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:01.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.489+0000 7f5bb3e00640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5b980a2b80 0x7f5b980a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009bd0 tx=0x7f5ba4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:01.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.491+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5bac0319a0 con 0x7f5bb4112760 2024-08-01T04:13:01.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.685+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bb41ab480 con 0x7f5bb4112760 2024-08-01T04:13:01.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:01 smithi003 ceph-mon[32262]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.685+0000 7f5bb1600640 1 -- 172.21.15.3:0/2523669695 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f5bac0bd1c0 con 0x7f5bb4112760 2024-08-01T04:13:01.687 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-08-01T04:13:01.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5b980a2b80 msgr2=0x7f5b980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5b980a2b80 0x7f5b980a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009bd0 tx=0x7f5ba4005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4112760 msgr2=0x7f5bb41a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4112760 0x7f5bb41a3000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5bac02aeb0 tx=0x7f5bac003f10 comp rx=0 tx=0).stop 2024-08-01T04:13:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 shutdown_connections 2024-08-01T04:13:01.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5b980a2b80 0x7f5b980a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb4112760 0x7f5bb41a3000 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5bb4076d70 0x7f5bb419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 --2- 172.21.15.3:0/2523669695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bb4076450 0x7f5bb419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 >> 172.21.15.3:0/2523669695 conn(0x7f5bb40ff7c0 msgr2=0x7f5bb4101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 shutdown_connections 2024-08-01T04:13:01.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:01.689+0000 7f5bbb200640 1 -- 172.21.15.3:0/2523669695 wait complete. 2024-08-01T04:13:01.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:01 smithi163 ceph-mon[37383]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:01.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:01 smithi160 ceph-mon[37224]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:02.481 INFO:tasks.cephadm:Adding mds.a on smithi003 2024-08-01T04:13:02.481 INFO:tasks.cephadm:Adding mds.d on smithi003 2024-08-01T04:13:02.481 INFO:tasks.cephadm:Adding mds.g on smithi003 2024-08-01T04:13:02.481 INFO:tasks.cephadm:Adding mds.j on smithi003 2024-08-01T04:13:02.481 INFO:tasks.cephadm:Adding mds.b on smithi160 2024-08-01T04:13:02.481 INFO:tasks.cephadm:Adding mds.e on smithi160 2024-08-01T04:13:02.482 INFO:tasks.cephadm:Adding mds.h on smithi160 2024-08-01T04:13:02.482 INFO:tasks.cephadm:Adding mds.k on smithi160 2024-08-01T04:13:02.482 INFO:tasks.cephadm:Adding mds.c on smithi163 2024-08-01T04:13:02.482 INFO:tasks.cephadm:Adding mds.f on smithi163 2024-08-01T04:13:02.482 INFO:tasks.cephadm:Adding mds.i on smithi163 2024-08-01T04:13:02.482 INFO:tasks.cephadm:Adding mds.l on smithi163 2024-08-01T04:13:02.482 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph orch apply mds all '12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l' 2024-08-01T04:13:02.747 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:13:02.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:02 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2523669695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:13:02.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:02 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2523669695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:13:02.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:02 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2523669695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:13:03.269 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 -- 172.21.15.163:0/2019148705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078101a00 msgr2=0x7fb078101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 --2- 172.21.15.163:0/2019148705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078101a00 0x7fb078101e80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009900 tx=0x7fb06c02a410 comp rx=0 tx=0).stop 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 -- 172.21.15.163:0/2019148705 shutdown_connections 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 --2- 172.21.15.163:0/2019148705 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0781023c0 0x7fb078108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 --2- 172.21.15.163:0/2019148705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078101a00 0x7fb078101e80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 --2- 172.21.15.163:0/2019148705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb078100800 0x7fb078100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 -- 172.21.15.163:0/2019148705 >> 172.21.15.163:0/2019148705 conn(0x7fb0780fbfb0 msgr2=0x7fb0780fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 -- 172.21.15.163:0/2019148705 shutdown_connections 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.246+0000 7fb07cef6640 1 -- 172.21.15.163:0/2019148705 wait complete. 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 Processor -- start 2024-08-01T04:13:03.270 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 -- start start 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 0x7fb078197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb078101a00 0x7fb078198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0781023c0 0x7fb07819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810c500 con 0x7fb0781023c0 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810c380 con 0x7fb078101a00 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07cef6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb07810c680 con 0x7fb078100800 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 0x7fb078197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 0x7fb078197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52494/0 (socket says 172.21.15.163:52494) 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.247+0000 7fb07659c640 1 -- 172.21.15.163:0/1099451173 learned_addr learned my addr 172.21.15.163:0/1099451173 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb075d9b640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb078101a00 0x7fb078198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:03.271 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07659c640 1 -- 172.21.15.163:0/1099451173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb078101a00 msgr2=0x7fb078198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07659c640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb078101a00 0x7fb078198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07659c640 1 -- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0781023c0 msgr2=0x7fb07819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07659c640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0781023c0 0x7fb07819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07659c640 1 -- 172.21.15.163:0/1099451173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07819f910 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07659c640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 0x7fb078197cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d950 tx=0x7fb06800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068017070 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb068009c30 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07819fc00 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.248+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0781a00f0 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.249+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068021060 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.249+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0781a0320 con 0x7fb078100800 2024-08-01T04:13:03.272 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.250+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb06801a070 con 0x7fb078100800 2024-08-01T04:13:03.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.251+0000 7fb0677fe640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb0600a2b80 0x7fb0600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:03.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.251+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fb0680ce730 con 0x7fb078100800 2024-08-01T04:13:03.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.251+0000 7fb075d9b640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb0600a2b80 0x7fb0600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:03.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.252+0000 7fb075d9b640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb0600a2b80 0x7fb0600a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb06c000c00 tx=0x7fb06c0315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:03.273 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.255+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb068009de0 con 0x7fb078100800 2024-08-01T04:13:03.442 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.440+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}) -- 0x7fb07806a580 con 0x7fb0600a2b80 2024-08-01T04:13:03.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.457+0000 7fb0677fe640 1 -- 172.21.15.163:0/1099451173 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb07806a580 con 0x7fb0600a2b80 2024-08-01T04:13:03.459 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mds.all update... 2024-08-01T04:13:03.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb0600a2b80 msgr2=0x7fb0600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:03.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb0600a2b80 0x7fb0600a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb06c000c00 tx=0x7fb06c0315d0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 msgr2=0x7fb078197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:03.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 0x7fb078197cf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d950 tx=0x7fb06800de10 comp rx=0 tx=0).stop 2024-08-01T04:13:03.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 shutdown_connections 2024-08-01T04:13:03.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb0600a2b80 0x7fb0600a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0781023c0 0x7fb07819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb078101a00 0x7fb078198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 --2- 172.21.15.163:0/1099451173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb078100800 0x7fb078197cf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:03.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 >> 172.21.15.163:0/1099451173 conn(0x7fb0780fbfb0 msgr2=0x7fb078069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:03.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 shutdown_connections 2024-08-01T04:13:03.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-08-01T04:13:03.460+0000 7fb07cef6640 1 -- 172.21.15.163:0/1099451173 wait complete. 2024-08-01T04:13:03.781 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:03 smithi163 ceph-mon[37383]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:03.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:03 smithi003 ceph-mon[32262]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:03.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:03 smithi160 ceph-mon[37224]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:04.156 DEBUG:teuthology.orchestra.run.smithi003:mds.a> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.a.service 2024-08-01T04:13:04.164 DEBUG:teuthology.orchestra.run.smithi003:mds.d> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.d.service 2024-08-01T04:13:04.211 DEBUG:teuthology.orchestra.run.smithi003:mds.g> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.g.service 2024-08-01T04:13:04.257 DEBUG:teuthology.orchestra.run.smithi003:mds.j> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.j.service 2024-08-01T04:13:04.304 DEBUG:teuthology.orchestra.run.smithi160:mds.b> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.b.service 2024-08-01T04:13:04.307 DEBUG:teuthology.orchestra.run.smithi160:mds.e> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.e.service 2024-08-01T04:13:04.349 DEBUG:teuthology.orchestra.run.smithi160:mds.h> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.h.service 2024-08-01T04:13:04.392 DEBUG:teuthology.orchestra.run.smithi160:mds.k> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.k.service 2024-08-01T04:13:04.435 DEBUG:teuthology.orchestra.run.smithi163:mds.c> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.c.service 2024-08-01T04:13:04.439 DEBUG:teuthology.orchestra.run.smithi163:mds.f> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.f.service 2024-08-01T04:13:04.482 DEBUG:teuthology.orchestra.run.smithi163:mds.i> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.i.service 2024-08-01T04:13:04.525 DEBUG:teuthology.orchestra.run.smithi163:mds.l> sudo journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.l.service 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:04.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:04.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:04.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:04.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:04 smithi003 ceph-mon[32262]: Deploying daemon mds.l on smithi163 2024-08-01T04:13:04.568 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-01T04:13:04.568 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:04.644 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:04.645 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:04.645 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:04.645 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:04.645 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:04 smithi163 ceph-mon[37383]: Deploying daemon mds.l on smithi163 2024-08-01T04:13:04.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:13:04.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-08-01T04:13:04.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:04.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:04.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:04 smithi160 ceph-mon[37224]: Deploying daemon mds.l on smithi163 2024-08-01T04:13:05.053 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 -- 172.21.15.3:0/421149354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d81125a0 msgr2=0x7fd5d8114990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/421149354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d81125a0 0x7fd5d8114990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009ca0 tx=0x7fd5d002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 -- 172.21.15.3:0/421149354 shutdown_connections 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/421149354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d81125a0 0x7fd5d8114990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/421149354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5d8076d70 0x7fd5d8112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/421149354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076450 0x7fd5d8076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 -- 172.21.15.3:0/421149354 >> 172.21.15.3:0/421149354 conn(0x7fd5d80ff620 msgr2=0x7fd5d8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:05.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 -- 172.21.15.3:0/421149354 shutdown_connections 2024-08-01T04:13:05.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.661+0000 7fd5d7e00640 1 -- 172.21.15.3:0/421149354 wait complete. 2024-08-01T04:13:05.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.662+0000 7fd5d7e00640 1 Processor -- start 2024-08-01T04:13:05.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 -- start start 2024-08-01T04:13:05.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 0x7fd5d819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:05.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076d70 0x7fd5d819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5d81125a0 0x7fd5d81a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8118140 con 0x7fd5d81125a0 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8117fc0 con 0x7fd5d8076d70 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81182c0 con 0x7fd5d8076450 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 0x7fd5d819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 0x7fd5d819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37648/0 (socket says 172.21.15.3:37648) 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d6a00640 1 -- 172.21.15.3:0/3871838350 learned_addr learned my addr 172.21.15.3:0/3871838350 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.663+0000 7fd5d7400640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5d81125a0 0x7fd5d81a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:05.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076d70 0x7fd5d819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6a00640 1 -- 172.21.15.3:0/3871838350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076d70 msgr2=0x7fd5d819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6a00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076d70 0x7fd5d819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6a00640 1 -- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5d81125a0 msgr2=0x7fd5d81a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6a00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5d81125a0 0x7fd5d81a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6a00640 1 -- 172.21.15.3:0/3871838350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d81a3590 con 0x7fd5d8076450 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076d70 0x7fd5d819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5d6a00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 0x7fd5d819e5e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9f0 tx=0x7fd5c800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.664+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800cd80 con 0x7fd5d8076450 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.665+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c8005500 con 0x7fd5d8076450 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.665+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8010690 con 0x7fd5d8076450 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.665+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d81a3880 con 0x7fd5d8076450 2024-08-01T04:13:05.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.665+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d81ab160 con 0x7fd5d8076450 2024-08-01T04:13:05.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.666+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d8076830 con 0x7fd5d8076450 2024-08-01T04:13:05.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.668+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd5c80046c0 con 0x7fd5d8076450 2024-08-01T04:13:05.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.670+0000 7fd5cf400640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:05.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.670+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fd5c80cf110 con 0x7fd5d8076450 2024-08-01T04:13:05.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.670+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:05.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.671+0000 7fd5d6000640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8078fc0 tx=0x7fd5c4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:05.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.673+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd5c809c470 con 0x7fd5d8076450 2024-08-01T04:13:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.911+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5d810d790 con 0x7fd5d8076450 2024-08-01T04:13:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.911+0000 7fd5cf400640 1 -- 172.21.15.3:0/3871838350 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fd5c809bd70 con 0x7fd5d8076450 2024-08-01T04:13:05.913 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:05.913 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"btime":"2024-08-01T04:05:46:214264+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-01T04:13:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 1 2024-08-01T04:13:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd5bc0a2c00 msgr2=0x7fd5bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8078fc0 tx=0x7fd5c4005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 msgr2=0x7fd5d819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 0x7fd5d819e5e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800e9f0 tx=0x7fd5c800edf0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 shutdown_connections 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd5bc0a2c00 0x7fd5bc0a50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5d81125a0 0x7fd5d81a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5d8076d70 0x7fd5d819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 --2- 172.21.15.3:0/3871838350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5d8076450 0x7fd5d819e5e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.916+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 >> 172.21.15.3:0/3871838350 conn(0x7fd5d80ff620 msgr2=0x7fd5d8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.917+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 shutdown_connections 2024-08-01T04:13:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:05.917+0000 7fd5d7e00640 1 -- 172.21.15.3:0/3871838350 wait complete. 2024-08-01T04:13:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:05 smithi163 ceph-mon[37383]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:05 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3871838350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:06.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:05 smithi003 ceph-mon[32262]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:06.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:05 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3871838350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:06.362 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:13:06 smithi163 podman[66917]: 2024-08-01 04:13:06.338652391 +0000 UTC m=+0.360369560 container start 5a666cacb5926c1fed6216a11f99dcf90042866a01b3405c44722b4dbc40e20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:13:06.362 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:13:06 smithi163 ceph-mds[66937]: -- 172.21.15.163:0/2143806473 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aaddf81860 con 0x55aaddf5e800 2024-08-01T04:13:06.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:05 smithi160 ceph-mon[37224]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:06.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:05 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3871838350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:06.796 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:13:06 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l[66933]: starting mds.l at 2024-08-01T04:13:06.796 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:13:06 smithi163 bash[66917]: 5a666cacb5926c1fed6216a11f99dcf90042866a01b3405c44722b4dbc40e20f 2024-08-01T04:13:06.796 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:13:06 smithi163 systemd[1]: Started Ceph mds.l for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:07.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.163:6832/4230398308,v1:172.21.15.163:6833/4230398308] up:boot 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: fsmap 1 up:standby 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:07.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:07 smithi160 ceph-mon[37224]: Deploying daemon mds.k on smithi160 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.163:6832/4230398308,v1:172.21.15.163:6833/4230398308] up:boot 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: fsmap 1 up:standby 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:07.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:07.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:07.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:07 smithi163 ceph-mon[37383]: Deploying daemon mds.k on smithi160 2024-08-01T04:13:07.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.163:6832/4230398308,v1:172.21.15.163:6833/4230398308] up:boot 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: fsmap 1 up:standby 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:07 smithi003 ceph-mon[32262]: Deploying daemon mds.k on smithi160 2024-08-01T04:13:08.652 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 systemd[1]: Starting Ceph mds.k for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:08.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:08.972 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 podman[66337]: 2024-08-01 04:13:08.623341713 +0000 UTC m=+0.017224483 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:08.972 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 podman[66337]: 2024-08-01 04:13:08.740618531 +0000 UTC m=+0.134501329 container create 83e17d5f6434875f320fd02adea1267fab889af745b59581aec5b666d1d344ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-01T04:13:08.973 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 podman[66337]: 2024-08-01 04:13:08.968334972 +0000 UTC m=+0.362217728 container init 83e17d5f6434875f320fd02adea1267fab889af745b59581aec5b666d1d344ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0) 2024-08-01T04:13:08.973 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 podman[66337]: 2024-08-01 04:13:08.971866127 +0000 UTC m=+0.365748883 container start 83e17d5f6434875f320fd02adea1267fab889af745b59581aec5b666d1d344ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-01T04:13:09.298 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:09.378 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 ceph-mds[66356]: -- 172.21.15.160:0/3480523954 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4abd9b860 con 0x55d4abd79000 2024-08-01T04:13:09.378 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:08 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k[66352]: starting mds.k at 2024-08-01T04:13:09.378 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:09 smithi160 bash[66337]: 83e17d5f6434875f320fd02adea1267fab889af745b59581aec5b666d1d344ca 2024-08-01T04:13:09.378 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:13:09 smithi160 systemd[1]: Started Ceph mds.k for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:09.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:09.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:09.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:09.647 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:09 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:09.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:09.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:09 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:09 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 -- 172.21.15.3:0/1389873417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c1028c0 msgr2=0x7f487c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 --2- 172.21.15.3:0/1389873417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c1028c0 0x7f487c102ca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02a490 comp rx=0 tx=0).stop 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 -- 172.21.15.3:0/1389873417 shutdown_connections 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 --2- 172.21.15.3:0/1389873417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c10bcf0 0x7f487c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 --2- 172.21.15.3:0/1389873417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1031e0 0x7f487c10b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 --2- 172.21.15.3:0/1389873417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c1028c0 0x7f487c102ca0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 -- 172.21.15.3:0/1389873417 >> 172.21.15.3:0/1389873417 conn(0x7f487c0fc360 msgr2=0x7f487c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 -- 172.21.15.3:0/1389873417 shutdown_connections 2024-08-01T04:13:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.899+0000 7f487be00640 1 -- 172.21.15.3:0/1389873417 wait complete. 2024-08-01T04:13:09.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.900+0000 7f487be00640 1 Processor -- start 2024-08-01T04:13:09.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.900+0000 7f487be00640 1 -- start start 2024-08-01T04:13:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.900+0000 7f487be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1028c0 0x7f487c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c1031e0 0x7f487c100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c10bcf0 0x7f487c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f487c111b40 con 0x7f487c10bcf0 2024-08-01T04:13:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487be00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1119c0 con 0x7f487c1028c0 2024-08-01T04:13:09.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f487c111cc0 con 0x7f487c1031e0 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c10bcf0 0x7f487c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1028c0 0x7f487c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c10bcf0 0x7f487c100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54924/0 (socket says 172.21.15.3:54924) 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487b400640 1 -- 172.21.15.3:0/611109769 learned_addr learned my addr 172.21.15.3:0/611109769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487a000640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c1031e0 0x7f487c100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487aa00640 1 -- 172.21.15.3:0/611109769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c1031e0 msgr2=0x7f487c100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487aa00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c1031e0 0x7f487c100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487aa00640 1 -- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c10bcf0 msgr2=0x7f487c100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487aa00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c10bcf0 0x7f487c100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487aa00640 1 -- 172.21.15.3:0/611109769 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c1a6460 con 0x7f487c1028c0 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.901+0000 7f487a000640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c1031e0 0x7f487c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.902+0000 7f487aa00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1028c0 0x7f487c100520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f486c02a9b0 tx=0x7f486c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:09.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.902+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c03f070 con 0x7f487c1028c0 2024-08-01T04:13:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.902+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c1a66f0 con 0x7f487c1028c0 2024-08-01T04:13:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.902+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486c031c90 con 0x7f487c1028c0 2024-08-01T04:13:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.902+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c035030 con 0x7f487c1028c0 2024-08-01T04:13:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.902+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1a6c20 con 0x7f487c1028c0 2024-08-01T04:13:09.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.904+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c102ca0 con 0x7f487c1028c0 2024-08-01T04:13:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.905+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f486c042070 con 0x7f487c1028c0 2024-08-01T04:13:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.907+0000 7f4873400640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48600a2c00 0x7f48600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.907+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f486c0ef4b0 con 0x7f487c1028c0 2024-08-01T04:13:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.907+0000 7f487a000640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48600a2c00 0x7f48600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.908+0000 7f487a000640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48600a2c00 0x7f48600a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f487c101fb0 tx=0x7f4868005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:09.911+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f486c0bc1c0 con 0x7f487c1028c0 2024-08-01T04:13:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.145+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f487c1a6e80 con 0x7f487c1028c0 2024-08-01T04:13:10.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.146+0000 7f4873400640 1 -- 172.21.15.3:0/611109769 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f486c0c04c0 con 0x7f487c1028c0 2024-08-01T04:13:10.147 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:10.147 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"btime":"2024-08-01T04:13:09:373963+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-01T04:13:10.147 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-08-01T04:13:10.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48600a2c00 msgr2=0x7f48600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:10.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48600a2c00 0x7f48600a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f487c101fb0 tx=0x7f4868005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:10.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1028c0 msgr2=0x7f487c100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:10.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1028c0 0x7f487c100520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f486c02a9b0 tx=0x7f486c003b20 comp rx=0 tx=0).stop 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 shutdown_connections 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48600a2c00 0x7f48600a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487c10bcf0 0x7f487c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f487c1031e0 0x7f487c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 --2- 172.21.15.3:0/611109769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f487c1028c0 0x7f487c100520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 >> 172.21.15.3:0/611109769 conn(0x7f487c0fc360 msgr2=0x7f487c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.148+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 shutdown_connections 2024-08-01T04:13:10.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:10.149+0000 7f487be00640 1 -- 172.21.15.3:0/611109769 wait complete. 2024-08-01T04:13:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:10 smithi160 ceph-mon[37224]: Deploying daemon mds.j on smithi003 2024-08-01T04:13:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:10 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6832/3012672686,v1:172.21.15.160:6833/3012672686] up:boot 2024-08-01T04:13:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:10 smithi160 ceph-mon[37224]: fsmap 2 up:standby 2024-08-01T04:13:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:10 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-01T04:13:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:10 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/611109769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:10 smithi163 ceph-mon[37383]: Deploying daemon mds.j on smithi003 2024-08-01T04:13:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:10 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6832/3012672686,v1:172.21.15.160:6833/3012672686] up:boot 2024-08-01T04:13:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:10 smithi163 ceph-mon[37383]: fsmap 2 up:standby 2024-08-01T04:13:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:10 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-01T04:13:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:10 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/611109769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:10 smithi003 ceph-mon[32262]: Deploying daemon mds.j on smithi003 2024-08-01T04:13:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:10 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6832/3012672686,v1:172.21.15.160:6833/3012672686] up:boot 2024-08-01T04:13:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:10 smithi003 ceph-mon[32262]: fsmap 2 up:standby 2024-08-01T04:13:10.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:10 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-01T04:13:10.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:10 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/611109769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:11 smithi160 ceph-mon[37224]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:11 smithi163 ceph-mon[37383]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:11.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:11 smithi003 ceph-mon[32262]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:12.715 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:12 smithi003 systemd[1]: Starting Ceph mds.j for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:13.017 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:12 smithi003 podman[71129]: 2024-08-01 04:13:12.617878285 +0000 UTC m=+0.033954668 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:13.018 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:12 smithi003 podman[71129]: 2024-08-01 04:13:12.727794471 +0000 UTC m=+0.143870879 container create cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-01T04:13:13.018 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:13 smithi003 podman[71129]: 2024-08-01 04:13:13.015280623 +0000 UTC m=+0.431357039 container init cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1) 2024-08-01T04:13:13.299 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:13 smithi003 podman[71129]: 2024-08-01 04:13:13.027133206 +0000 UTC m=+0.443209588 container start cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-01T04:13:13.299 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mds[71147]: -- 172.21.15.3:0/1735552430 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa80025c20 con 0x55aa80002800 2024-08-01T04:13:13.299 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j[71143]: starting mds.j at 2024-08-01T04:13:13.299 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:13 smithi003 bash[71129]: cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 2024-08-01T04:13:13.299 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:13:13 smithi003 systemd[1]: Started Ceph mds.j for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:13.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:13 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:13.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:13.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:13 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:13.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:13.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:13.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:13 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:14.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:14 smithi003 ceph-mon[32262]: Deploying daemon mds.d on smithi003 2024-08-01T04:13:14.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:14 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.3:6834/3318532700,v1:172.21.15.3:6835/3318532700] up:boot 2024-08-01T04:13:14.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:14 smithi003 ceph-mon[32262]: fsmap 3 up:standby 2024-08-01T04:13:14.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:14 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-01T04:13:14.671 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:14 smithi163 ceph-mon[37383]: Deploying daemon mds.d on smithi003 2024-08-01T04:13:14.671 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:14 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.3:6834/3318532700,v1:172.21.15.3:6835/3318532700] up:boot 2024-08-01T04:13:14.671 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:14 smithi163 ceph-mon[37383]: fsmap 3 up:standby 2024-08-01T04:13:14.671 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:14 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-01T04:13:14.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:14 smithi160 ceph-mon[37224]: Deploying daemon mds.d on smithi003 2024-08-01T04:13:14.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:14 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.3:6834/3318532700,v1:172.21.15.3:6835/3318532700] up:boot 2024-08-01T04:13:14.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:14 smithi160 ceph-mon[37224]: fsmap 3 up:standby 2024-08-01T04:13:14.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:14 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-01T04:13:15.377 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:15.771 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:15 smithi003 ceph-mon[32262]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:15.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:15 smithi163 ceph-mon[37383]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:15.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:15 smithi160 ceph-mon[37224]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:16.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 -- 172.21.15.3:0/1085932065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0102090 msgr2=0x7f6ed010e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/1085932065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0102090 0x7f6ed010e5c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009cd0 tx=0x7f6ec802a8f0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 -- 172.21.15.3:0/1085932065 shutdown_connections 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/1085932065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0102090 0x7f6ed010e5c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/1085932065 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/1085932065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01076f0 0x7f6ed0107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 -- 172.21.15.3:0/1085932065 >> 172.21.15.3:0/1085932065 conn(0x7f6ed00fd470 msgr2=0x7f6ed00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 -- 172.21.15.3:0/1085932065 shutdown_connections 2024-08-01T04:13:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.055+0000 7f6ed4c00640 1 -- 172.21.15.3:0/1085932065 wait complete. 2024-08-01T04:13:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.056+0000 7f6ed4c00640 1 Processor -- start 2024-08-01T04:13:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.056+0000 7f6ed4c00640 1 -- start start 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ed4c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ed4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed0102090 0x7f6ed0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ed4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01076f0 0x7f6ed0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ed4c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111b40 con 0x7f6ed01076f0 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ed4c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01119c0 con 0x7f6ed01016f0 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ed4c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111cc0 con 0x7f6ed0102090 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ecf400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ecf400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:55290/0 (socket says 172.21.15.3:55290) 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ecf400640 1 -- 172.21.15.3:0/194859995 learned_addr learned my addr 172.21.15.3:0/194859995 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.057+0000 7f6ecea00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed0102090 0x7f6ed0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01076f0 0x7f6ed0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecf400640 1 -- 172.21.15.3:0/194859995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed0102090 msgr2=0x7f6ed0074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecf400640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed0102090 0x7f6ed0074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecf400640 1 -- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01076f0 msgr2=0x7f6ed0074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecf400640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01076f0 0x7f6ed0074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecf400640 1 -- 172.21.15.3:0/194859995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed01a6460 con 0x7f6ed01016f0 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01076f0 0x7f6ed0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.058+0000 7f6ecf400640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0074220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000d950 tx=0x7f6ec000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.059+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0017030 con 0x7f6ed01016f0 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.059+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec0004020 con 0x7f6ed01016f0 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.059+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00047a0 con 0x7f6ed01016f0 2024-08-01T04:13:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.059+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed01a6750 con 0x7f6ed01016f0 2024-08-01T04:13:16.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.059+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01a6c40 con 0x7f6ed01016f0 2024-08-01T04:13:16.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.060+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0101b50 con 0x7f6ed01016f0 2024-08-01T04:13:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.062+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6ec00041d0 con 0x7f6ed01016f0 2024-08-01T04:13:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.063+0000 7f6ec7e00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6eb40a2c00 0x7f6eb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.064+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f6ec00ce550 con 0x7f6ed01016f0 2024-08-01T04:13:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.064+0000 7f6ecea00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6eb40a2c00 0x7f6eb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.065+0000 7f6ecea00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6eb40a2c00 0x7f6eb40a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0075cb0 tx=0x7f6ebc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:16.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.068+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6ec009b8b0 con 0x7f6ed01016f0 2024-08-01T04:13:16.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.320+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ed0109c60 con 0x7f6ed01016f0 2024-08-01T04:13:16.323 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:16 smithi003 systemd[1]: Starting Ceph mds.d for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.321+0000 7f6ec7e00640 1 -- 172.21.15.3:0/194859995 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f6ec009b1b0 con 0x7f6ed01016f0 2024-08-01T04:13:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:16.324 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"btime":"2024-08-01T04:13:13:397589+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-08-01T04:13:16.324 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 4 2024-08-01T04:13:16.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6eb40a2c00 msgr2=0x7f6eb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:16.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6eb40a2c00 0x7f6eb40a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0075cb0 tx=0x7f6ebc005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:16.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 msgr2=0x7f6ed0074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:16.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0074220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000d950 tx=0x7f6ec000de10 comp rx=0 tx=0).stop 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 shutdown_connections 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6eb40a2c00 0x7f6eb40a50c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01076f0 0x7f6ed0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed0102090 0x7f6ed0074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 --2- 172.21.15.3:0/194859995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed01016f0 0x7f6ed0074220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.324+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 >> 172.21.15.3:0/194859995 conn(0x7f6ed00fd470 msgr2=0x7f6ed00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.325+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 shutdown_connections 2024-08-01T04:13:16.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:16.325+0000 7f6ed4c00640 1 -- 172.21.15.3:0/194859995 wait complete. 2024-08-01T04:13:16.721 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:16 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/194859995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:16.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:16 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/194859995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:16.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:16 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/194859995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:17.049 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:16 smithi003 podman[71536]: 2024-08-01 04:13:16.757764112 +0000 UTC m=+0.174195460 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:17.424 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 podman[71536]: 2024-08-01 04:13:17.256117921 +0000 UTC m=+0.672549300 container create aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:13:17.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:17 smithi163 ceph-mon[37383]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:17.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:17 smithi003 ceph-mon[32262]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:17.799 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 podman[71536]: 2024-08-01 04:13:17.534835569 +0000 UTC m=+0.951267014 container init aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:13:17.799 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 podman[71536]: 2024-08-01 04:13:17.544932681 +0000 UTC m=+0.961364060 container start aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=) 2024-08-01T04:13:17.799 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 ceph-mds[71558]: -- 172.21.15.3:0/3783666482 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f8294df860 con 0x55f8294bd400 2024-08-01T04:13:17.800 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d[71554]: starting mds.d at 2024-08-01T04:13:17.800 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 bash[71536]: aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 2024-08-01T04:13:17.800 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:13:17 smithi003 systemd[1]: Started Ceph mds.d for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:17 smithi160 ceph-mon[37224]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:19.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:19.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:19.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:19.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: Deploying daemon mds.e on smithi160 2024-08-01T04:13:19.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.3:6836/258713238,v1:172.21.15.3:6837/258713238] up:boot 2024-08-01T04:13:19.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: fsmap 4 up:standby 2024-08-01T04:13:19.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: Deploying daemon mds.e on smithi160 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.3:6836/258713238,v1:172.21.15.3:6837/258713238] up:boot 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: fsmap 4 up:standby 2024-08-01T04:13:19.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-01T04:13:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: Deploying daemon mds.e on smithi160 2024-08-01T04:13:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.3:6836/258713238,v1:172.21.15.3:6837/258713238] up:boot 2024-08-01T04:13:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: fsmap 4 up:standby 2024-08-01T04:13:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-01T04:13:19.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:19.676 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:19.745 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:19 smithi160 systemd[1]: Starting Ceph mds.e for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:20.031 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:19 smithi003 ceph-mon[32262]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:20.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:19 smithi163 ceph-mon[37383]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:20.055 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:19 smithi160 ceph-mon[37224]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:20.056 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:19 smithi160 podman[66650]: 2024-08-01 04:13:19.794136904 +0000 UTC m=+0.113789258 container create 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:13:20.056 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:19 smithi160 podman[66650]: 2024-08-01 04:13:19.698267064 +0000 UTC m=+0.017919432 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:20.056 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:20 smithi160 podman[66650]: 2024-08-01 04:13:20.055094139 +0000 UTC m=+0.374746503 container init 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_CLEAN=True, ceph=True) 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 -- 172.21.15.3:0/293900911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90112780 msgr2=0x7f2a90114b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 --2- 172.21.15.3:0/293900911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90112780 0x7f2a90114b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009ca0 tx=0x7f2a8802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 -- 172.21.15.3:0/293900911 shutdown_connections 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 --2- 172.21.15.3:0/293900911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90112780 0x7f2a90114b70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 --2- 172.21.15.3:0/293900911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a90076d70 0x7f2a90112240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 --2- 172.21.15.3:0/293900911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a90076450 0x7f2a90076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 -- 172.21.15.3:0/293900911 >> 172.21.15.3:0/293900911 conn(0x7f2a900ff780 msgr2=0x7f2a90101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:20.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 -- 172.21.15.3:0/293900911 shutdown_connections 2024-08-01T04:13:20.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.353+0000 7f2a96000640 1 -- 172.21.15.3:0/293900911 wait complete. 2024-08-01T04:13:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.354+0000 7f2a96000640 1 Processor -- start 2024-08-01T04:13:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 -- start start 2024-08-01T04:13:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a90076450 0x7f2a901a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:20.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 0x7f2a901a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a90112780 0x7f2a9019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90118320 con 0x7f2a90076450 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901181a0 con 0x7f2a90112780 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a96000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901184a0 con 0x7f2a90076d70 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a8ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 0x7f2a901a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a8ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 0x7f2a901a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53228/0 (socket says 172.21.15.3:53228) 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a8ea00640 1 -- 172.21.15.3:0/2133558428 learned_addr learned my addr 172.21.15.3:0/2133558428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a8f400640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a90076450 0x7f2a901a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.355+0000 7f2a8fe00640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a90112780 0x7f2a9019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a8ea00640 1 -- 172.21.15.3:0/2133558428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a90112780 msgr2=0x7f2a9019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a8ea00640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a90112780 0x7f2a9019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a8ea00640 1 -- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a90076450 msgr2=0x7f2a901a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a8ea00640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a90076450 0x7f2a901a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a8ea00640 1 -- 172.21.15.3:0/2133558428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9019ef40 con 0x7f2a90076d70 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a8ea00640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 0x7f2a901a4af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9f0 tx=0x7f2a7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800cd80 con 0x7f2a90076d70 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a78010910 con 0x7f2a90076d70 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9019f230 con 0x7f2a90076d70 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78018990 con 0x7f2a90076d70 2024-08-01T04:13:20.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.356+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a901ab5a0 con 0x7f2a90076d70 2024-08-01T04:13:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.358+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a90077b90 con 0x7f2a90076d70 2024-08-01T04:13:20.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.360+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2a78010ac0 con 0x7f2a90076d70 2024-08-01T04:13:20.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.361+0000 7f2a87e00640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2a700a2c00 0x7f2a700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.361+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f2a780cee10 con 0x7f2a90076d70 2024-08-01T04:13:20.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.361+0000 7f2a8f400640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2a700a2c00 0x7f2a700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:20.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.362+0000 7f2a8f400640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2a700a2c00 0x7f2a700a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009bd0 tx=0x7f2a80005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:20.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.365+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2a780d2030 con 0x7f2a90076d70 2024-08-01T04:13:20.402 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:20 smithi160 podman[66650]: 2024-08-01 04:13:20.05792453 +0000 UTC m=+0.377576885 container start 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-01T04:13:20.402 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:20 smithi160 ceph-mds[66670]: -- 172.21.15.160:0/2214846322 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a26bed860 con 0x563a26bca800 2024-08-01T04:13:20.402 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:20 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e[66665]: starting mds.e at 2024-08-01T04:13:20.403 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:20 smithi160 bash[66650]: 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f 2024-08-01T04:13:20.403 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:13:20 smithi160 systemd[1]: Started Ceph mds.e for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.599+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a9010d910 con 0x7f2a90076d70 2024-08-01T04:13:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.600+0000 7f2a87e00640 1 -- 172.21.15.3:0/2133558428 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f2a7809bb20 con 0x7f2a90076d70 2024-08-01T04:13:20.602 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:20.602 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"btime":"2024-08-01T04:13:18:422697+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-08-01T04:13:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 5 2024-08-01T04:13:20.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.603+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2a700a2c00 msgr2=0x7f2a700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.603+0000 7f2a96000640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2a700a2c00 0x7f2a700a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009bd0 tx=0x7f2a80005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.603+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 msgr2=0x7f2a901a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.603+0000 7f2a96000640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 0x7f2a901a4af0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800e9f0 tx=0x7f2a7800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.603+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 shutdown_connections 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.603+0000 7f2a96000640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f2a700a2c00 0x7f2a700a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.604+0000 7f2a96000640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a90112780 0x7f2a9019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.604+0000 7f2a96000640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a90076d70 0x7f2a901a4af0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.604+0000 7f2a96000640 1 --2- 172.21.15.3:0/2133558428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a90076450 0x7f2a901a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.604+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 >> 172.21.15.3:0/2133558428 conn(0x7f2a900ff780 msgr2=0x7f2a90101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.604+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 shutdown_connections 2024-08-01T04:13:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:20.604+0000 7f2a96000640 1 -- 172.21.15.3:0/2133558428 wait complete. 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:21.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:21.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: Deploying daemon mds.f on smithi163 2024-08-01T04:13:21.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2133558428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:21.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6834/3179303046,v1:172.21.15.160:6835/3179303046] up:boot 2024-08-01T04:13:21.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: fsmap 5 up:standby 2024-08-01T04:13:21.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:21 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: Deploying daemon mds.f on smithi163 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2133558428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6834/3179303046,v1:172.21.15.160:6835/3179303046] up:boot 2024-08-01T04:13:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: fsmap 5 up:standby 2024-08-01T04:13:21.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:21 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:13:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: Deploying daemon mds.f on smithi163 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2133558428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6834/3179303046,v1:172.21.15.160:6835/3179303046] up:boot 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: fsmap 5 up:standby 2024-08-01T04:13:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:21 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:13:22.296 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 systemd[1]: Starting Ceph mds.f for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:22.708 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 podman[67226]: 2024-08-01 04:13:22.338276258 +0000 UTC m=+0.017414913 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:22.708 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 podman[67226]: 2024-08-01 04:13:22.445785189 +0000 UTC m=+0.124923834 container create 6b97cc6642de49d9583fd6dbcd8171ee91de6ad49e29aa304aa85c8955d147ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-01T04:13:22.708 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 podman[67226]: 2024-08-01 04:13:22.681798056 +0000 UTC m=+0.360936701 container init 6b97cc6642de49d9583fd6dbcd8171ee91de6ad49e29aa304aa85c8955d147ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-01T04:13:22.708 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 podman[67226]: 2024-08-01 04:13:22.684317692 +0000 UTC m=+0.363456338 container start 6b97cc6642de49d9583fd6dbcd8171ee91de6ad49e29aa304aa85c8955d147ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-01T04:13:22.708 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 ceph-mds[67245]: -- 172.21.15.163:0/3579284364 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a75f85f860 con 0x55a75f83c800 2024-08-01T04:13:23.045 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f[67241]: starting mds.f at 2024-08-01T04:13:23.046 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 bash[67226]: 6b97cc6642de49d9583fd6dbcd8171ee91de6ad49e29aa304aa85c8955d147ec 2024-08-01T04:13:23.046 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:13:22 smithi163 systemd[1]: Started Ceph mds.f for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:23.567 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: Deploying daemon mds.c on smithi163 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.163:6834/2143574012,v1:172.21.15.163:6835/2143574012] up:boot 2024-08-01T04:13:24.023 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: fsmap 6 up:standby 2024-08-01T04:13:24.024 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:23 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-01T04:13:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: Deploying daemon mds.c on smithi163 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.163:6834/2143574012,v1:172.21.15.163:6835/2143574012] up:boot 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: fsmap 6 up:standby 2024-08-01T04:13:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:23 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-01T04:13:24.059 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: Deploying daemon mds.c on smithi163 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.163:6834/2143574012,v1:172.21.15.163:6835/2143574012] up:boot 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: fsmap 6 up:standby 2024-08-01T04:13:24.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:23 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-01T04:13:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.744+0000 7fdd9f000640 1 -- 172.21.15.3:0/9441305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074b60 msgr2=0x7fdd9810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.744+0000 7fdd9f000640 1 --2- 172.21.15.3:0/9441305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074b60 0x7fdd9810dc30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009900 tx=0x7fdd8002a480 comp rx=0 tx=0).stop 2024-08-01T04:13:24.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 -- 172.21.15.3:0/9441305 shutdown_connections 2024-08-01T04:13:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 --2- 172.21.15.3:0/9441305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd9810e170 0x7fdd98110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 --2- 172.21.15.3:0/9441305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074b60 0x7fdd9810dc30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 --2- 172.21.15.3:0/9441305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd98074240 0x7fdd98074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 -- 172.21.15.3:0/9441305 >> 172.21.15.3:0/9441305 conn(0x7fdd980fd610 msgr2=0x7fdd980ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 -- 172.21.15.3:0/9441305 shutdown_connections 2024-08-01T04:13:24.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.745+0000 7fdd9f000640 1 -- 172.21.15.3:0/9441305 wait complete. 2024-08-01T04:13:24.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.746+0000 7fdd9f000640 1 Processor -- start 2024-08-01T04:13:24.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.746+0000 7fdd9f000640 1 -- start start 2024-08-01T04:13:24.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9f000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 0x7fdd9819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:24.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9f000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 0x7fdd9819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:24.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9f000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9810e170 0x7fdd9819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9f000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98113eb0 con 0x7fdd98074b60 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9f000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98113d30 con 0x7fdd9810e170 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9f000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98114030 con 0x7fdd98074240 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd97400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 0x7fdd9819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 0x7fdd9819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd97e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 0x7fdd9819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:59758/0 (socket says 172.21.15.3:59758) 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd97400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 0x7fdd9819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42232/0 (socket says 172.21.15.3:42232) 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd97e00640 1 -- 172.21.15.3:0/1086666847 learned_addr learned my addr 172.21.15.3:0/1086666847 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:24.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.747+0000 7fdd9d000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9810e170 0x7fdd9819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:24.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97e00640 1 -- 172.21.15.3:0/1086666847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9810e170 msgr2=0x7fdd9819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:24.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97e00640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9810e170 0x7fdd9819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97e00640 1 -- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 msgr2=0x7fdd9819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97e00640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 0x7fdd9819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97e00640 1 -- 172.21.15.3:0/1086666847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9819f360 con 0x7fdd98074240 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97400640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 0x7fdd9819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd9d000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9810e170 0x7fdd9819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.748+0000 7fdd97e00640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 0x7fdd9819a3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800e9f0 tx=0x7fdd8800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.749+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8800cd80 con 0x7fdd98074240 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.749+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd88010910 con 0x7fdd98074240 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.749+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88018990 con 0x7fdd98074240 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.749+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9819f650 con 0x7fdd98074240 2024-08-01T04:13:24.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.749+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a6f30 con 0x7fdd98074240 2024-08-01T04:13:24.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.750+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd98075980 con 0x7fdd98074240 2024-08-01T04:13:24.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.752+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdd88010ac0 con 0x7fdd98074240 2024-08-01T04:13:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.753+0000 7fdd94c00640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdd7c0a2b80 0x7fdd7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:24.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.754+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fdd880cef50 con 0x7fdd98074240 2024-08-01T04:13:24.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.754+0000 7fdd97400640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdd7c0a2b80 0x7fdd7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:24.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.755+0000 7fdd97400640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdd7c0a2b80 0x7fdd7c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd80031510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:24.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.757+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdd8809bc60 con 0x7fdd98074240 2024-08-01T04:13:24.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.991+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd981092f0 con 0x7fdd98074240 2024-08-01T04:13:24.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.992+0000 7fdd94c00640 1 -- 172.21.15.3:0/1086666847 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7fdd8809ff60 con 0x7fdd98074240 2024-08-01T04:13:24.995 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:24.995 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-08-01T04:13:23:212045+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-01T04:13:24.995 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-08-01T04:13:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.995+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdd7c0a2b80 msgr2=0x7fdd7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.995+0000 7fdd9f000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdd7c0a2b80 0x7fdd7c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000c00 tx=0x7fdd80031510 comp rx=0 tx=0).stop 2024-08-01T04:13:24.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.995+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 msgr2=0x7fdd9819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.995+0000 7fdd9f000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 0x7fdd9819a3b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800e9f0 tx=0x7fdd8800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 shutdown_connections 2024-08-01T04:13:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdd7c0a2b80 0x7fdd7c0a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9810e170 0x7fdd9819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd98074b60 0x7fdd9819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 --2- 172.21.15.3:0/1086666847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd98074240 0x7fdd9819a3b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 >> 172.21.15.3:0/1086666847 conn(0x7fdd980fd610 msgr2=0x7fdd980fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 shutdown_connections 2024-08-01T04:13:24.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:24.996+0000 7fdd9f000640 1 -- 172.21.15.3:0/1086666847 wait complete. 2024-08-01T04:13:25.046 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:24 smithi163 systemd[1]: Starting Ceph mds.c for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:25.404 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 podman[67527]: 2024-08-01 04:13:25.017438818 +0000 UTC m=+0.017198621 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:25.404 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 podman[67527]: 2024-08-01 04:13:25.142177605 +0000 UTC m=+0.141937399 container create 9eae5ad1084f43f65e7b0fa8f7eb122d3c89a079f5d328bb32c5e1a32efda9e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:13:25.404 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 podman[67527]: 2024-08-01 04:13:25.403119174 +0000 UTC m=+0.402878967 container init 9eae5ad1084f43f65e7b0fa8f7eb122d3c89a079f5d328bb32c5e1a32efda9e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2) 2024-08-01T04:13:25.796 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 podman[67527]: 2024-08-01 04:13:25.40618452 +0000 UTC m=+0.405944322 container start 9eae5ad1084f43f65e7b0fa8f7eb122d3c89a079f5d328bb32c5e1a32efda9e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-01T04:13:25.796 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mds[67546]: -- 172.21.15.163:0/4039120689 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c55369860 con 0x561c55347000 2024-08-01T04:13:25.796 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c[67542]: starting mds.c at 2024-08-01T04:13:25.796 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 bash[67527]: 9eae5ad1084f43f65e7b0fa8f7eb122d3c89a079f5d328bb32c5e1a32efda9e8 2024-08-01T04:13:25.796 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:13:25 smithi163 systemd[1]: Started Ceph mds.c for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:26.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:26.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1086666847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:25 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1086666847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:26.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:25 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:26.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1086666847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:26.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:25 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:27.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:26 smithi160 ceph-mon[37224]: Deploying daemon mds.h on smithi160 2024-08-01T04:13:27.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:26 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.163:6836/1698078935,v1:172.21.15.163:6837/1698078935] up:boot 2024-08-01T04:13:27.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:26 smithi160 ceph-mon[37224]: fsmap 7 up:standby 2024-08-01T04:13:27.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:26 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:13:27.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:26 smithi163 ceph-mon[37383]: Deploying daemon mds.h on smithi160 2024-08-01T04:13:27.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:26 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.163:6836/1698078935,v1:172.21.15.163:6837/1698078935] up:boot 2024-08-01T04:13:27.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:26 smithi163 ceph-mon[37383]: fsmap 7 up:standby 2024-08-01T04:13:27.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:26 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:13:27.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:26 smithi003 ceph-mon[32262]: Deploying daemon mds.h on smithi160 2024-08-01T04:13:27.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:26 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.163:6836/1698078935,v1:172.21.15.163:6837/1698078935] up:boot 2024-08-01T04:13:27.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:26 smithi003 ceph-mon[32262]: fsmap 7 up:standby 2024-08-01T04:13:27.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:26 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:13:27.821 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:27 smithi160 systemd[1]: Starting Ceph mds.h for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:27.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:28.093 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:27 smithi160 ceph-mon[37224]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:28.094 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:27 smithi160 podman[66952]: 2024-08-01 04:13:27.72212778 +0000 UTC m=+0.017041748 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:28.094 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:27 smithi160 podman[66952]: 2024-08-01 04:13:27.840202556 +0000 UTC m=+0.135116526 container create 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-01T04:13:28.094 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:28 smithi160 podman[66952]: 2024-08-01 04:13:28.093133625 +0000 UTC m=+0.388047601 container init 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-01T04:13:28.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:27 smithi163 ceph-mon[37383]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:28.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:27 smithi003 ceph-mon[32262]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:28.402 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:28 smithi160 podman[66952]: 2024-08-01 04:13:28.095842728 +0000 UTC m=+0.390756703 container start 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1) 2024-08-01T04:13:28.403 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:28 smithi160 ceph-mds[66972]: -- 172.21.15.160:0/4065753757 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561d2b5d5860 con 0x561d2b5b3400 2024-08-01T04:13:28.403 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:28 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h[66968]: starting mds.h at 2024-08-01T04:13:28.403 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:28 smithi160 bash[66952]: 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 2024-08-01T04:13:28.403 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:13:28 smithi160 systemd[1]: Started Ceph mds.h for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:28.421 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.090+0000 7ff2db000640 1 -- 172.21.15.3:0/3552796575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d4103900 msgr2=0x7ff2d4103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.090+0000 7ff2db000640 1 --2- 172.21.15.3:0/3552796575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d4103900 0x7ff2d4103d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009930 tx=0x7ff2c002a420 comp rx=0 tx=0).stop 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 -- 172.21.15.3:0/3552796575 shutdown_connections 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 --2- 172.21.15.3:0/3552796575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d41042a0 0x7ff2d41107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 --2- 172.21.15.3:0/3552796575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d4103900 0x7ff2d4103d60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 --2- 172.21.15.3:0/3552796575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4109900 0x7ff2d4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 -- 172.21.15.3:0/3552796575 >> 172.21.15.3:0/3552796575 conn(0x7ff2d40ff640 msgr2=0x7ff2d4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:29.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 -- 172.21.15.3:0/3552796575 shutdown_connections 2024-08-01T04:13:29.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.091+0000 7ff2db000640 1 -- 172.21.15.3:0/3552796575 wait complete. 2024-08-01T04:13:29.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.092+0000 7ff2db000640 1 Processor -- start 2024-08-01T04:13:29.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.092+0000 7ff2db000640 1 -- start start 2024-08-01T04:13:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2db000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4103900 0x7ff2d419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2db000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d41042a0 0x7ff2d419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2db000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 0x7ff2d419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2db000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4113d00 con 0x7ff2d41042a0 2024-08-01T04:13:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2db000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4113b80 con 0x7ff2d4109900 2024-08-01T04:13:29.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2db000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4113e80 con 0x7ff2d4103900 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2d9200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d41042a0 0x7ff2d419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2da600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 0x7ff2d419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2d9c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4103900 0x7ff2d419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2d9200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d41042a0 0x7ff2d419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42250/0 (socket says 172.21.15.3:42250) 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2da600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 0x7ff2d419eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:36686/0 (socket says 172.21.15.3:36686) 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.093+0000 7ff2d9200640 1 -- 172.21.15.3:0/1051680851 learned_addr learned my addr 172.21.15.3:0/1051680851 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2da600640 1 -- 172.21.15.3:0/1051680851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4103900 msgr2=0x7ff2d419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2da600640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4103900 0x7ff2d419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2da600640 1 -- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d41042a0 msgr2=0x7ff2d419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2da600640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d41042a0 0x7ff2d419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2da600640 1 -- 172.21.15.3:0/1051680851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d419f1e0 con 0x7ff2d4109900 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2d9c00640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4103900 0x7ff2d419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2da600640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 0x7ff2d419eb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00da10 tx=0x7ff2cc00ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc00bd70 con 0x7ff2d4109900 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2cc003e50 con 0x7ff2d4109900 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0106a0 con 0x7ff2d4109900 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d419f4d0 con 0x7ff2d4109900 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.094+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a6db0 con 0x7ff2d4109900 2024-08-01T04:13:29.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.096+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d4105040 con 0x7ff2d4109900 2024-08-01T04:13:29.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.098+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff2cc003970 con 0x7ff2d4109900 2024-08-01T04:13:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.099+0000 7ff2d2000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.099+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff2cc0cefa0 con 0x7ff2d4109900 2024-08-01T04:13:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.100+0000 7ff2d9c00640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.100+0000 7ff2d9c00640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009f40 tx=0x7ff2c8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:29.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.103+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff2cc09c300 con 0x7ff2d4109900 2024-08-01T04:13:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.326+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2d402d5e0 con 0x7ff2d4109900 2024-08-01T04:13:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.327+0000 7ff2d2000640 1 -- 172.21.15.3:0/1051680851 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7ff2cc09bc00 con 0x7ff2d4109900 2024-08-01T04:13:29.329 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:29.329 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"btime":"2024-08-01T04:13:28:862510+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-01T04:13:29.330 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2024-08-01T04:13:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff2bc0a2c00 msgr2=0x7ff2bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009f40 tx=0x7ff2c8005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 msgr2=0x7ff2d419eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 0x7ff2d419eb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00da10 tx=0x7ff2cc00ded0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 shutdown_connections 2024-08-01T04:13:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff2bc0a2c00 0x7ff2bc0a50c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2d4109900 0x7ff2d419eb00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2d41042a0 0x7ff2d419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 --2- 172.21.15.3:0/1051680851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2d4103900 0x7ff2d419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.329+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 >> 172.21.15.3:0/1051680851 conn(0x7ff2d40ff640 msgr2=0x7ff2d40fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.330+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 shutdown_connections 2024-08-01T04:13:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:29.330+0000 7ff2db000640 1 -- 172.21.15.3:0/1051680851 wait complete. 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: Deploying daemon mds.i on smithi163 2024-08-01T04:13:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] up:boot 2024-08-01T04:13:29.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: fsmap 8 up:standby 2024-08-01T04:13:29.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:29 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: Deploying daemon mds.i on smithi163 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] up:boot 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: fsmap 8 up:standby 2024-08-01T04:13:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:29 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:13:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: Deploying daemon mds.i on smithi163 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] up:boot 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: fsmap 8 up:standby 2024-08-01T04:13:29.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:29 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:13:30.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:30 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1051680851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:30.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:30 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1051680851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:30.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:30 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1051680851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:31.046 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:13:30 smithi163 systemd[1]: Started Ceph mds.i for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:31.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: Deploying daemon mds.g on smithi003 2024-08-01T04:13:31.963 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.163:6838/4057809310,v1:172.21.15.163:6839/4057809310] up:boot 2024-08-01T04:13:31.963 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: fsmap 9 up:standby 2024-08-01T04:13:31.963 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:31 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:32.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: Deploying daemon mds.g on smithi003 2024-08-01T04:13:32.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.163:6838/4057809310,v1:172.21.15.163:6839/4057809310] up:boot 2024-08-01T04:13:32.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: fsmap 9 up:standby 2024-08-01T04:13:32.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:31 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: Deploying daemon mds.g on smithi003 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.163:6838/4057809310,v1:172.21.15.163:6839/4057809310] up:boot 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: fsmap 9 up:standby 2024-08-01T04:13:32.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:31 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:13:32.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:33.152 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:34.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:33 smithi003 ceph-mon[32262]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.061+0000 7f334ec00640 1 -- 172.21.15.3:0/1319925945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348112760 msgr2=0x7f3348114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:34.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 --2- 172.21.15.3:0/1319925945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348112760 0x7f3348114b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3340009900 tx=0x7f334002a410 comp rx=0 tx=0).stop 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 -- 172.21.15.3:0/1319925945 shutdown_connections 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 --2- 172.21.15.3:0/1319925945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348112760 0x7f3348114b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 --2- 172.21.15.3:0/1319925945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348076d70 0x7f3348112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 --2- 172.21.15.3:0/1319925945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076450 0x7f3348076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 -- 172.21.15.3:0/1319925945 >> 172.21.15.3:0/1319925945 conn(0x7f33480ff7c0 msgr2=0x7f3348101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 -- 172.21.15.3:0/1319925945 shutdown_connections 2024-08-01T04:13:34.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.062+0000 7f334ec00640 1 -- 172.21.15.3:0/1319925945 wait complete. 2024-08-01T04:13:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.063+0000 7f334ec00640 1 Processor -- start 2024-08-01T04:13:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.063+0000 7f334ec00640 1 -- start start 2024-08-01T04:13:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.063+0000 7f334ec00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 0x7f33481a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.063+0000 7f334ec00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076d70 0x7f33481a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f334ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348112760 0x7f334819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:34.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f334ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3348118300 con 0x7f3348112760 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f334ec00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3348118180 con 0x7f3348076d70 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f334ec00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3348118480 con 0x7f3348076450 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f334cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348112760 0x7f334819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 0x7f33481a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076d70 0x7f33481a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 0x7f33481a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:57518/0 (socket says 172.21.15.3:57518) 2024-08-01T04:13:34.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347e00640 1 -- 172.21.15.3:0/787770439 learned_addr learned my addr 172.21.15.3:0/787770439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:34.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347400640 1 -- 172.21.15.3:0/787770439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 msgr2=0x7f33481a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:34.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347400640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 0x7f33481a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347400640 1 -- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348112760 msgr2=0x7f334819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:34.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347400640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348112760 0x7f334819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347400640 1 -- 172.21.15.3:0/787770439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334819ed00 con 0x7f3348076d70 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f3347e00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 0x7f33481a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.064+0000 7f334cc00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348112760 0x7f334819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.065+0000 7f3347400640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076d70 0x7f33481a4950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9f0 tx=0x7f333400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.065+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333400ccc0 con 0x7f3348076d70 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.065+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334819eff0 con 0x7f3348076d70 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.065+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3334010910 con 0x7f3348076d70 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.065+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334018960 con 0x7f3348076d70 2024-08-01T04:13:34.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.065+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33481ab2e0 con 0x7f3348076d70 2024-08-01T04:13:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.067+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3348076830 con 0x7f3348076d70 2024-08-01T04:13:34.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.068+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3334010ac0 con 0x7f3348076d70 2024-08-01T04:13:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.070+0000 7f3344c00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f332c0a2c00 0x7f332c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.071+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f33340ceef0 con 0x7f3348076d70 2024-08-01T04:13:34.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.071+0000 7f3347e00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f332c0a2c00 0x7f332c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:34.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.071+0000 7f3347e00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f332c0a2c00 0x7f332c0a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f33380049d0 tx=0x7f3338009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:34.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.074+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f333409bc00 con 0x7f3348076d70 2024-08-01T04:13:34.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:33 smithi160 ceph-mon[37224]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:34.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:33 smithi163 ceph-mon[37383]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:34.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.325+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f334810d8f0 con 0x7f3348076d70 2024-08-01T04:13:34.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.326+0000 7f3344c00640 1 -- 172.21.15.3:0/787770439 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7f333409ff00 con 0x7f3348076d70 2024-08-01T04:13:34.328 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:34.328 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"btime":"2024-08-01T04:13:31:247782+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-01T04:13:34.328 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 10 2024-08-01T04:13:34.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f332c0a2c00 msgr2=0x7f332c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:34.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f332c0a2c00 0x7f332c0a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f33380049d0 tx=0x7f3338009500 comp rx=0 tx=0).stop 2024-08-01T04:13:34.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076d70 msgr2=0x7f33481a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:34.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076d70 0x7f33481a4950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9f0 tx=0x7f333400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 shutdown_connections 2024-08-01T04:13:34.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f332c0a2c00 0x7f332c0a50c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3348112760 0x7f334819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3348076d70 0x7f33481a4950 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 --2- 172.21.15.3:0/787770439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3348076450 0x7f33481a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:34.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 >> 172.21.15.3:0/787770439 conn(0x7f33480ff7c0 msgr2=0x7f3348101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:34.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 shutdown_connections 2024-08-01T04:13:34.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:34.329+0000 7f334ec00640 1 -- 172.21.15.3:0/787770439 wait complete. 2024-08-01T04:13:35.135 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:34 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/787770439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:35.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:34 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/787770439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:35.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:34 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/787770439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:36.139 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:36.140 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:35 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:36.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:36.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:36.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:36.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:35 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:36.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:35 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:37.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:36 smithi003 ceph-mon[32262]: Deploying daemon mds.a on smithi003 2024-08-01T04:13:37.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:36 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.3:6838/367445798,v1:172.21.15.3:6839/367445798] up:boot 2024-08-01T04:13:37.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:36 smithi003 ceph-mon[32262]: fsmap 10 up:standby 2024-08-01T04:13:37.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:36 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-01T04:13:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:36 smithi160 ceph-mon[37224]: Deploying daemon mds.a on smithi003 2024-08-01T04:13:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:36 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.3:6838/367445798,v1:172.21.15.3:6839/367445798] up:boot 2024-08-01T04:13:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:36 smithi160 ceph-mon[37224]: fsmap 10 up:standby 2024-08-01T04:13:37.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:36 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-01T04:13:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:36 smithi163 ceph-mon[37383]: Deploying daemon mds.a on smithi003 2024-08-01T04:13:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:36 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.3:6838/367445798,v1:172.21.15.3:6839/367445798] up:boot 2024-08-01T04:13:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:36 smithi163 ceph-mon[37383]: fsmap 10 up:standby 2024-08-01T04:13:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:36 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-01T04:13:37.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:37.919 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:37 smithi003 ceph-mon[32262]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:38.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:37 smithi160 ceph-mon[37224]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:38.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:37 smithi163 ceph-mon[37383]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:38.298 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:37 smithi003 systemd[1]: Starting Ceph mds.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:38.614 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 podman[72626]: 2024-08-01 04:13:38.24083763 +0000 UTC m=+0.035921112 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:38.614 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 podman[72626]: 2024-08-01 04:13:38.355861913 +0000 UTC m=+0.150945410 container create e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-01T04:13:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:38.898 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 podman[72626]: 2024-08-01 04:13:38.610081117 +0000 UTC m=+0.405164644 container init e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724) 2024-08-01T04:13:38.898 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 podman[72626]: 2024-08-01 04:13:38.623297731 +0000 UTC m=+0.418381210 container start e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-08-01T04:13:38.898 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 ceph-mds[72671]: -- 172.21.15.3:0/1618727854 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f49b71b860 con 0x55f49b6f9000 2024-08-01T04:13:38.898 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a[72667]: starting mds.a at 2024-08-01T04:13:38.899 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 bash[72626]: e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f 2024-08-01T04:13:38.899 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:13:38 smithi003 systemd[1]: Started Ceph mds.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:39.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.440+0000 7f7ef1e00640 1 -- 172.21.15.3:0/1265175072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 msgr2=0x7f7eec109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.440+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/1265175072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec109ce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009900 tx=0x7f7ee002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 -- 172.21.15.3:0/1265175072 shutdown_connections 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/1265175072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7eec1042a0 0x7f7eec1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/1265175072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec103900 0x7f7eec103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/1265175072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec109ce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 -- 172.21.15.3:0/1265175072 >> 172.21.15.3:0/1265175072 conn(0x7f7eec0ff600 msgr2=0x7f7eec101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 -- 172.21.15.3:0/1265175072 shutdown_connections 2024-08-01T04:13:39.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.441+0000 7f7ef1e00640 1 -- 172.21.15.3:0/1265175072 wait complete. 2024-08-01T04:13:39.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.442+0000 7f7ef1e00640 1 Processor -- start 2024-08-01T04:13:39.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.442+0000 7f7ef1e00640 1 -- start start 2024-08-01T04:13:39.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7eec103900 0x7f7eec19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec1042a0 0x7f7eec19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec113d50 con 0x7f7eec1042a0 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec113bd0 con 0x7f7eec109900 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec113ed0 con 0x7f7eec103900 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7eec103900 0x7f7eec19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec1042a0 0x7f7eec19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec1042a0 0x7f7eec19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53606/0 (socket says 172.21.15.3:53606) 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7eebe00640 1 -- 172.21.15.3:0/860010773 learned_addr learned my addr 172.21.15.3:0/860010773 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:39.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1400640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1400640 1 -- 172.21.15.3:0/860010773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7eec103900 msgr2=0x7f7eec19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1400640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7eec103900 0x7f7eec19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1400640 1 -- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec1042a0 msgr2=0x7f7eec19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1400640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec1042a0 0x7f7eec19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.443+0000 7f7ef1400640 1 -- 172.21.15.3:0/860010773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec19f1e0 con 0x7f7eec109900 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.444+0000 7f7ef1400640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec19eb00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009fd0 tx=0x7f7ee400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.444+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400ce20 con 0x7f7eec109900 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.444+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee4005020 con 0x7f7eec109900 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.444+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4010720 con 0x7f7eec109900 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.444+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec19f4d0 con 0x7f7eec109900 2024-08-01T04:13:39.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.444+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec1a6db0 con 0x7f7eec109900 2024-08-01T04:13:39.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.445+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec103d60 con 0x7f7eec109900 2024-08-01T04:13:39.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.447+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7ee4005540 con 0x7f7eec109900 2024-08-01T04:13:39.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.449+0000 7f7ee9600640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:39.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.449+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f7ee40cf130 con 0x7f7eec109900 2024-08-01T04:13:39.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.449+0000 7f7ef0a00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:39.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.450+0000 7f7ef0a00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee0005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:39.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.453+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ee409c490 con 0x7f7eec109900 2024-08-01T04:13:39.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.686+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eec02d5e0 con 0x7f7eec109900 2024-08-01T04:13:39.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.687+0000 7f7ee9600640 1 -- 172.21.15.3:0/860010773 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7f7ee409bd90 con 0x7f7eec109900 2024-08-01T04:13:39.690 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:39.690 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"btime":"2024-08-01T04:13:38:888008+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-08-01T04:13:39.690 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 12 2024-08-01T04:13:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7ed40a2c00 msgr2=0x7f7ed40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee0005b50 comp rx=0 tx=0).stop 2024-08-01T04:13:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 msgr2=0x7f7eec19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec19eb00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009fd0 tx=0x7f7ee400ef10 comp rx=0 tx=0).stop 2024-08-01T04:13:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 shutdown_connections 2024-08-01T04:13:39.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7ed40a2c00 0x7f7ed40a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7eec109900 0x7f7eec19eb00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7eec1042a0 0x7f7eec19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 --2- 172.21.15.3:0/860010773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7eec103900 0x7f7eec19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 >> 172.21.15.3:0/860010773 conn(0x7f7eec0ff600 msgr2=0x7f7eec100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 shutdown_connections 2024-08-01T04:13:39.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:39.690+0000 7f7ef1e00640 1 -- 172.21.15.3:0/860010773 wait complete. 2024-08-01T04:13:40.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:40.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.047 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: Deploying daemon mds.b on smithi160 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] up:boot 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: fsmap 11 up:standby 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:13:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:39 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/860010773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: Deploying daemon mds.b on smithi160 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] up:boot 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: fsmap 11 up:standby 2024-08-01T04:13:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:13:40.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:39 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/860010773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: Deploying daemon mds.b on smithi160 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] up:boot 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: fsmap 11 up:standby 2024-08-01T04:13:40.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:13:40.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:39 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/860010773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:41.131 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:40 smithi160 systemd[1]: Starting Ceph mds.b for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:13:41.402 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 podman[67267]: 2024-08-01 04:13:41.03242206 +0000 UTC m=+0.017217233 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:13:41.402 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 podman[67267]: 2024-08-01 04:13:41.153391813 +0000 UTC m=+0.138187011 container create 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:13:41.839 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 podman[67267]: 2024-08-01 04:13:41.448540164 +0000 UTC m=+0.433335343 container init 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-01T04:13:41.839 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 podman[67267]: 2024-08-01 04:13:41.451794502 +0000 UTC m=+0.436589659 container start 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:13:41.839 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mds[67286]: -- 172.21.15.160:0/2837963829 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a82e1d860 con 0x559a82dfb000 2024-08-01T04:13:41.839 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: starting mds.b at 2024-08-01T04:13:41.839 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 bash[67267]: 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 2024-08-01T04:13:41.839 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:41 smithi160 systemd[1]: Started Ceph mds.b for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:13:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mon[37224]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:41 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:41 smithi163 ceph-mon[37383]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:41 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:41 smithi003 ceph-mon[32262]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:41 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:42.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:13:43.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:42 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] up:boot 2024-08-01T04:13:43.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:42 smithi160 ceph-mon[37224]: fsmap 12 up:standby 2024-08-01T04:13:43.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:42 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:43.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:42 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:13:43.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:42 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] up:boot 2024-08-01T04:13:43.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:42 smithi163 ceph-mon[37383]: fsmap 12 up:standby 2024-08-01T04:13:43.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:42 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:43.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:42 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:13:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:42 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] up:boot 2024-08-01T04:13:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:42 smithi003 ceph-mon[32262]: fsmap 12 up:standby 2024-08-01T04:13:43.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:42 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:43.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:42 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:13:43.523 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:44.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:43 smithi003 ceph-mon[32262]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:44.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:43 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:43 smithi160 ceph-mon[37224]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:44.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:43 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 -- 172.21.15.3:0/193599996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f177410df30 msgr2=0x7f1774110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 --2- 172.21.15.3:0/193599996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f177410df30 0x7f1774110380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f176c009ca0 tx=0x7f176c02a950 comp rx=0 tx=0).stop 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 -- 172.21.15.3:0/193599996 shutdown_connections 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 --2- 172.21.15.3:0/193599996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f177410df30 0x7f1774110380 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 --2- 172.21.15.3:0/193599996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074c70 0x7f177410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 --2- 172.21.15.3:0/193599996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1774074350 0x7f1774074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.160+0000 7f177a800640 1 -- 172.21.15.3:0/193599996 >> 172.21.15.3:0/193599996 conn(0x7f17740fd410 msgr2=0x7f17740ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:44.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.161+0000 7f177a800640 1 -- 172.21.15.3:0/193599996 shutdown_connections 2024-08-01T04:13:44.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.161+0000 7f177a800640 1 -- 172.21.15.3:0/193599996 wait complete. 2024-08-01T04:13:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 Processor -- start 2024-08-01T04:13:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 -- start start 2024-08-01T04:13:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 0x7f177419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1774074c70 0x7f177419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f177410df30 0x7f177419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:44.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1774113d30 con 0x7f1774074350 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1774113bb0 con 0x7f177410df30 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.162+0000 7f177a800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1774113eb0 con 0x7f1774074c70 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1779400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 0x7f177419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1778a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1774074c70 0x7f177419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1779400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 0x7f177419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36872/0 (socket says 172.21.15.3:36872) 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1779400640 1 -- 172.21.15.3:0/1811029877 learned_addr learned my addr 172.21.15.3:0/1811029877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1779e00640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f177410df30 0x7f177419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1778a00640 1 -- 172.21.15.3:0/1811029877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f177410df30 msgr2=0x7f177419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1778a00640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f177410df30 0x7f177419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1778a00640 1 -- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 msgr2=0x7f177419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:44.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1778a00640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 0x7f177419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1778a00640 1 -- 172.21.15.3:0/1811029877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177419f140 con 0x7f1774074c70 2024-08-01T04:13:44.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.163+0000 7f1779400640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 0x7f177419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.164+0000 7f1778a00640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1774074c70 0x7f177419a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f176800e9f0 tx=0x7f176800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.164+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800ccc0 con 0x7f1774074c70 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.164+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1768010910 con 0x7f1774074c70 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.164+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177419f430 con 0x7f1774074c70 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.164+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768018960 con 0x7f1774074c70 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.164+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17741a6d10 con 0x7f1774074c70 2024-08-01T04:13:44.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.165+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1774074730 con 0x7f1774074c70 2024-08-01T04:13:44.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.167+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1768010ac0 con 0x7f1774074c70 2024-08-01T04:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.169+0000 7f1772000640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f175c0a2b80 0x7f175c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.169+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f17680cef60 con 0x7f1774074c70 2024-08-01T04:13:44.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.169+0000 7f1779400640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f175c0a2b80 0x7f175c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:44.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.170+0000 7f1779400640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f175c0a2b80 0x7f175c0a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1764011500 tx=0x7f1764005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:44.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.176+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f176809bc70 con 0x7f1774074c70 2024-08-01T04:13:44.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:43 smithi163 ceph-mon[37383]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:44.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:43 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.428+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17741a7040 con 0x7f1774074c70 2024-08-01T04:13:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.429+0000 7f1772000640 1 -- 172.21.15.3:0/1811029877 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f176809ff70 con 0x7f1774074c70 2024-08-01T04:13:44.431 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:44.431 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"btime":"2024-08-01T04:13:41:835421+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24548,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-01T04:13:44.432 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 13 2024-08-01T04:13:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f175c0a2b80 msgr2=0x7f175c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f175c0a2b80 0x7f175c0a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1764011500 tx=0x7f1764005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1774074c70 msgr2=0x7f177419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1774074c70 0x7f177419a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f176800e9f0 tx=0x7f176800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 shutdown_connections 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f175c0a2b80 0x7f175c0a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f177410df30 0x7f177419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1774074c70 0x7f177419a6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 --2- 172.21.15.3:0/1811029877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1774074350 0x7f177419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 >> 172.21.15.3:0/1811029877 conn(0x7f17740fd410 msgr2=0x7f17740fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 shutdown_connections 2024-08-01T04:13:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:44.432+0000 7f177a800640 1 -- 172.21.15.3:0/1811029877 wait complete. 2024-08-01T04:13:45.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:44 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1811029877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:45.197 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:44 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1811029877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:45.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:44 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1811029877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:13:45.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-01T04:13:45.797 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:46.145 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:46.145 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.145 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.146 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:46.146 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:46.146 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.146 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:45 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:46.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:46.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:45 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:46.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:46.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:46.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:46.245 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:46.246 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:45 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:13:46.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 -- 172.21.15.3:0/579730821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 msgr2=0x7fbe481142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 --2- 172.21.15.3:0/579730821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe481142a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009890 tx=0x7fbe3402a360 comp rx=0 tx=0).stop 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 -- 172.21.15.3:0/579730821 shutdown_connections 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 --2- 172.21.15.3:0/579730821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe481147e0 0x7fbe48116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 --2- 172.21.15.3:0/579730821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe481142a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 --2- 172.21.15.3:0/579730821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe4806e220 0x7fbe4806e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 -- 172.21.15.3:0/579730821 >> 172.21.15.3:0/579730821 conn(0x7fbe48072eb0 msgr2=0x7fbe480732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 -- 172.21.15.3:0/579730821 shutdown_connections 2024-08-01T04:13:46.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.399+0000 7fbe4de00640 1 -- 172.21.15.3:0/579730821 wait complete. 2024-08-01T04:13:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.400+0000 7fbe4de00640 1 Processor -- start 2024-08-01T04:13:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 -- start start 2024-08-01T04:13:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4806e220 0x7fbe4810a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe4810a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:46.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe481147e0 0x7fbe4810ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4811a280 con 0x7fbe4806e220 2024-08-01T04:13:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4811a100 con 0x7fbe4806eb40 2024-08-01T04:13:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4de00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4811a400 con 0x7fbe481147e0 2024-08-01T04:13:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe4810a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe4810a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:49148/0 (socket says 172.21.15.3:49148) 2024-08-01T04:13:46.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe47e00640 1 -- 172.21.15.3:0/3266924290 learned_addr learned my addr 172.21.15.3:0/3266924290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.401+0000 7fbe4ca00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4806e220 0x7fbe4810a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe4d400640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe481147e0 0x7fbe4810ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe47e00640 1 -- 172.21.15.3:0/3266924290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe481147e0 msgr2=0x7fbe4810ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe47e00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe481147e0 0x7fbe4810ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe47e00640 1 -- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4806e220 msgr2=0x7fbe4810a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe47e00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4806e220 0x7fbe4810a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe47e00640 1 -- 172.21.15.3:0/3266924290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe481aed40 con 0x7fbe4806eb40 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe4ca00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4806e220 0x7fbe4810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:46.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.402+0000 7fbe47e00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe4810a640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe34033ae0 tx=0x7fbe34033b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.403+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34037040 con 0x7fbe4806eb40 2024-08-01T04:13:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.403+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe34040070 con 0x7fbe4806eb40 2024-08-01T04:13:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.403+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe481aefd0 con 0x7fbe4806eb40 2024-08-01T04:13:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.403+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481af430 con 0x7fbe4806eb40 2024-08-01T04:13:46.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.403+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34003960 con 0x7fbe4806eb40 2024-08-01T04:13:46.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.404+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4806e600 con 0x7fbe4806eb40 2024-08-01T04:13:46.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.406+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbe34030580 con 0x7fbe4806eb40 2024-08-01T04:13:46.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.407+0000 7fbe45600640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe300a2c00 0x7fbe300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:46.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.408+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fbe340eeb60 con 0x7fbe4806eb40 2024-08-01T04:13:46.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.408+0000 7fbe4ca00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe300a2c00 0x7fbe300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:46.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.408+0000 7fbe4ca00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe300a2c00 0x7fbe300a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0096c0 tx=0x7fbe3c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:46.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.412+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbe340bbec0 con 0x7fbe4806eb40 2024-08-01T04:13:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.630+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbe4810f960 con 0x7fbe4806eb40 2024-08-01T04:13:46.633 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:46.633 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-08-01T04:13:46.633 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:13:46.633 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-08-01T04:13:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-08-01T04:13:46.634 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:13:46.634 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-08-01T04:13:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.630+0000 7fbe45600640 1 -- 172.21.15.3:0/3266924290 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7fbe340bb7c0 con 0x7fbe4806eb40 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe300a2c00 msgr2=0x7fbe300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe300a2c00 0x7fbe300a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0096c0 tx=0x7fbe3c009260 comp rx=0 tx=0).stop 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 msgr2=0x7fbe4810a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe4810a640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe34033ae0 tx=0x7fbe34033b10 comp rx=0 tx=0).stop 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 shutdown_connections 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbe300a2c00 0x7fbe300a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe481147e0 0x7fbe4810ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe4806eb40 0x7fbe4810a640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 --2- 172.21.15.3:0/3266924290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe4806e220 0x7fbe4810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.633+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 >> 172.21.15.3:0/3266924290 conn(0x7fbe48072eb0 msgr2=0x7fbe4806c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.634+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 shutdown_connections 2024-08-01T04:13:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:46.634+0000 7fbe4de00640 1 -- 172.21.15.3:0/3266924290 wait complete. 2024-08-01T04:13:47.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:46 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3266924290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-01T04:13:47.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:46 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3266924290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-01T04:13:47.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:46 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3266924290' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-01T04:13:48.177 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-01T04:13:48.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-01T04:13:48.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:48 smithi163 ceph-mon[37383]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:48.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:48 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:48 smithi003 ceph-mon[32262]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:48.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:48 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:48 smithi160 ceph-mon[37224]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:48.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:48 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:49.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 -- 172.21.15.3:0/2880525469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae4102240 msgr2=0x7f4ae410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:49.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 --2- 172.21.15.3:0/2880525469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae4102240 0x7f4ae410e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009cd0 tx=0x7f4adc02a8e0 comp rx=0 tx=0).stop 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 -- 172.21.15.3:0/2880525469 shutdown_connections 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 --2- 172.21.15.3:0/2880525469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae4102240 0x7f4ae410e770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 --2- 172.21.15.3:0/2880525469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae41018a0 0x7f4ae4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 --2- 172.21.15.3:0/2880525469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41078a0 0x7f4ae4107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 -- 172.21.15.3:0/2880525469 >> 172.21.15.3:0/2880525469 conn(0x7f4ae40fd5c0 msgr2=0x7f4ae40ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 -- 172.21.15.3:0/2880525469 shutdown_connections 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.370+0000 7f4aebe00640 1 -- 172.21.15.3:0/2880525469 wait complete. 2024-08-01T04:13:49.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.371+0000 7f4aebe00640 1 Processor -- start 2024-08-01T04:13:49.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 -- start start 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 0x7f4ae4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae4102240 0x7f4ae4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41078a0 0x7f4ae419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111cf0 con 0x7f4ae41078a0 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111b70 con 0x7f4ae41018a0 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4aebe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111e70 con 0x7f4ae4102240 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4ae9400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 0x7f4ae4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4ae9400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 0x7f4ae4198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:49168/0 (socket says 172.21.15.3:49168) 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.372+0000 7f4ae9400640 1 -- 172.21.15.3:0/3780997931 learned_addr learned my addr 172.21.15.3:0/3780997931 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9e00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41078a0 0x7f4ae419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:49.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae8a00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae4102240 0x7f4ae4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:49.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9400640 1 -- 172.21.15.3:0/3780997931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae4102240 msgr2=0x7f4ae4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:49.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9400640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae4102240 0x7f4ae4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:49.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9400640 1 -- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41078a0 msgr2=0x7f4ae419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:49.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9400640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41078a0 0x7f4ae419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:49.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9400640 1 -- 172.21.15.3:0/3780997931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae419d100 con 0x7f4ae41018a0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae8a00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae4102240 0x7f4ae4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.373+0000 7f4ae9400640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 0x7f4ae4198150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800ea60 tx=0x7f4ad800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.374+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800cde0 con 0x7f4ae41018a0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.374+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad80035f0 con 0x7f4ae41018a0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.374+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8010670 con 0x7f4ae41018a0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.374+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae419d3f0 con 0x7f4ae41018a0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.374+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae41a4cd0 con 0x7f4ae41018a0 2024-08-01T04:13:49.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.375+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae4101d00 con 0x7f4ae41018a0 2024-08-01T04:13:49.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.378+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4ad80037a0 con 0x7f4ae41018a0 2024-08-01T04:13:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.379+0000 7f4ae2000640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4acc0a2c00 0x7f4acc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.379+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f4ad80ceee0 con 0x7f4ae41018a0 2024-08-01T04:13:49.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.380+0000 7f4ae8a00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4acc0a2c00 0x7f4acc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:49.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.380+0000 7f4ae8a00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4acc0a2c00 0x7f4acc0a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40ffb10 tx=0x7f4ad0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:49.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.381+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4ad809c240 con 0x7f4ae41018a0 2024-08-01T04:13:49.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:49.585+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4ae41a4fc0 con 0x7f4ae41018a0 2024-08-01T04:13:50.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:50 smithi160 ceph-mon[37224]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:50.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:50.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:50.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:50 smithi003 ceph-mon[32262]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:50.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:50.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.443+0000 7f4ae2000640 1 -- 172.21.15.3:0/3780997931 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f4ad809bb40 con 0x7f4ae41018a0 2024-08-01T04:13:50.448 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-08-01T04:13:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.449+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4acc0a2c00 msgr2=0x7f4acc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.449+0000 7f4aebe00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4acc0a2c00 0x7f4acc0a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40ffb10 tx=0x7f4ad0005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.449+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 msgr2=0x7f4ae4198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.449+0000 7f4aebe00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 0x7f4ae4198150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800ea60 tx=0x7f4ad800ef20 comp rx=0 tx=0).stop 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.451+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 shutdown_connections 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.451+0000 7f4aebe00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4acc0a2c00 0x7f4acc0a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.451+0000 7f4aebe00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ae41078a0 0x7f4ae419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.451+0000 7f4aebe00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ae4102240 0x7f4ae4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.451+0000 7f4aebe00640 1 --2- 172.21.15.3:0/3780997931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ae41018a0 0x7f4ae4198150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.451+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 >> 172.21.15.3:0/3780997931 conn(0x7f4ae40fd5c0 msgr2=0x7f4ae40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.452+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 shutdown_connections 2024-08-01T04:13:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:50.452+0000 7f4aebe00640 1 -- 172.21.15.3:0/3780997931 wait complete. 2024-08-01T04:13:50.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:50 smithi163 ceph-mon[37383]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:50.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:50.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:51.302 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-01T04:13:51.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3780997931' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-01T04:13:51.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-01T04:13:51.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:51.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:51.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:51.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:51.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-01T04:13:51.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:51 smithi160 ceph-mon[37224]: osdmap e80: 12 total, 12 up, 12 in 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3780997931' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-01T04:13:51.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:51 smithi163 ceph-mon[37383]: osdmap e80: 12 total, 12 up, 12 in 2024-08-01T04:13:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3780997931' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-01T04:13:51.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:51 smithi003 ceph-mon[32262]: osdmap e80: 12 total, 12 up, 12 in 2024-08-01T04:13:51.791 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:52.498 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:52 smithi003 ceph-mon[32262]: osdmap e81: 12 total, 12 up, 12 in 2024-08-01T04:13:52.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.495+0000 7f1658e00640 1 -- 172.21.15.3:0/1348388466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076450 msgr2=0x7f1654076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:52.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.495+0000 7f1658e00640 1 --2- 172.21.15.3:0/1348388466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076450 0x7f1654076830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1644009900 tx=0x7f164402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:13:52.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 -- 172.21.15.3:0/1348388466 shutdown_connections 2024-08-01T04:13:52.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 --2- 172.21.15.3:0/1348388466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16541125a0 0x7f1654114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 --2- 172.21.15.3:0/1348388466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1654076d70 0x7f1654112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 --2- 172.21.15.3:0/1348388466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076450 0x7f1654076830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 -- 172.21.15.3:0/1348388466 >> 172.21.15.3:0/1348388466 conn(0x7f16540ff640 msgr2=0x7f1654101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 -- 172.21.15.3:0/1348388466 shutdown_connections 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.496+0000 7f1658e00640 1 -- 172.21.15.3:0/1348388466 wait complete. 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 Processor -- start 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 -- start start 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 0x7f165419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:52.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076d70 0x7f165419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16541125a0 0x7f16541a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1654118180 con 0x7f1654076450 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1654118000 con 0x7f16541125a0 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.497+0000 7f1658e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1654118300 con 0x7f1654076d70 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 0x7f165419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 0x7f165419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33164/0 (socket says 172.21.15.3:33164) 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 -- 172.21.15.3:0/2705576010 learned_addr learned my addr 172.21.15.3:0/2705576010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16541125a0 0x7f16541a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1652a00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076d70 0x7f165419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 -- 172.21.15.3:0/2705576010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076d70 msgr2=0x7f165419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076d70 0x7f165419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 -- 172.21.15.3:0/2705576010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16541125a0 msgr2=0x7f16541a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:52.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16541125a0 0x7f16541a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653400640 1 -- 172.21.15.3:0/2705576010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16541a3630 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.498+0000 7f1653e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16541125a0 0x7f16541a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.499+0000 7f1653400640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 0x7f165419e680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1644004770 tx=0x7f1644003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.499+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164403f070 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.499+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1644031ca0 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.499+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16541a38c0 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.499+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644035030 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.499+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541ab200 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.500+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1654077b90 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.503+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1644042070 con 0x7f1654076450 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.505+0000 7f164be00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16380a2b80 0x7f16380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:52.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.505+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f16440efc10 con 0x7f1654076450 2024-08-01T04:13:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.505+0000 7f1652a00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16380a2b80 0x7f16380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.506+0000 7f1652a00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16380a2b80 0x7f16380a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1654079030 tx=0x7f163c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:52.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.508+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f16440bce00 con 0x7f1654076450 2024-08-01T04:13:52.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:52.702+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f165410d730 con 0x7f1654076450 2024-08-01T04:13:52.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:52 smithi163 ceph-mon[37383]: osdmap e81: 12 total, 12 up, 12 in 2024-08-01T04:13:52.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:52 smithi160 ceph-mon[37224]: osdmap e81: 12 total, 12 up, 12 in 2024-08-01T04:13:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.471+0000 7f164be00640 1 -- 172.21.15.3:0/2705576010 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f16440bc700 con 0x7f1654076450 2024-08-01T04:13:53.474 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-08-01T04:13:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.478+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16380a2b80 msgr2=0x7f16380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.478+0000 7f1658e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16380a2b80 0x7f16380a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1654079030 tx=0x7f163c005f90 comp rx=0 tx=0).stop 2024-08-01T04:13:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.478+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 msgr2=0x7f165419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.478+0000 7f1658e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 0x7f165419e680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1644004770 tx=0x7f1644003c10 comp rx=0 tx=0).stop 2024-08-01T04:13:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.479+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 shutdown_connections 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.479+0000 7f1658e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f16380a2b80 0x7f16380a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.479+0000 7f1658e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16541125a0 0x7f16541a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.479+0000 7f1658e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1654076d70 0x7f165419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.479+0000 7f1658e00640 1 --2- 172.21.15.3:0/2705576010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1654076450 0x7f165419e680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.479+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 >> 172.21.15.3:0/2705576010 conn(0x7f16540ff640 msgr2=0x7f1654101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.480+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 shutdown_connections 2024-08-01T04:13:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:53.480+0000 7f1658e00640 1 -- 172.21.15.3:0/2705576010 wait complete. 2024-08-01T04:13:53.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:53 smithi163 ceph-mon[37383]: pgmap v283: 33 pgs: 9 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:53.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:53 smithi163 ceph-mon[37383]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-01T04:13:53.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:53 smithi163 ceph-mon[37383]: osdmap e82: 12 total, 12 up, 12 in 2024-08-01T04:13:53.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:53 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2705576010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-01T04:13:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:53 smithi003 ceph-mon[32262]: pgmap v283: 33 pgs: 9 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:53 smithi003 ceph-mon[32262]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-01T04:13:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:53 smithi003 ceph-mon[32262]: osdmap e82: 12 total, 12 up, 12 in 2024-08-01T04:13:53.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:53 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2705576010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-01T04:13:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:53 smithi160 ceph-mon[37224]: pgmap v283: 33 pgs: 9 creating+peering, 20 unknown, 4 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:53 smithi160 ceph-mon[37224]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-01T04:13:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:53 smithi160 ceph-mon[37224]: osdmap e82: 12 total, 12 up, 12 in 2024-08-01T04:13:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:53 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2705576010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-01T04:13:54.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-01T04:13:54.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:54 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2705576010' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-01T04:13:54.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:54 smithi003 ceph-mon[32262]: osdmap e83: 12 total, 12 up, 12 in 2024-08-01T04:13:54.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:54 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2705576010' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-01T04:13:54.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:54 smithi163 ceph-mon[37383]: osdmap e83: 12 total, 12 up, 12 in 2024-08-01T04:13:54.826 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:54.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:54 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2705576010' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-01T04:13:54.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:54 smithi160 ceph-mon[37224]: osdmap e83: 12 total, 12 up, 12 in 2024-08-01T04:13:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.516+0000 7feeea800640 1 -- 172.21.15.3:0/3056491824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee4108180 msgr2=0x7feee4108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.516+0000 7feeea800640 1 --2- 172.21.15.3:0/3056491824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee4108180 0x7feee4108560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:13:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 -- 172.21.15.3:0/3056491824 shutdown_connections 2024-08-01T04:13:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 --2- 172.21.15.3:0/3056491824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee410d260 0x7feee4114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 --2- 172.21.15.3:0/3056491824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108b30 0x7feee410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:55.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 --2- 172.21.15.3:0/3056491824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee4108180 0x7feee4108560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 -- 172.21.15.3:0/3056491824 >> 172.21.15.3:0/3056491824 conn(0x7feee40778d0 msgr2=0x7feee4079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 -- 172.21.15.3:0/3056491824 shutdown_connections 2024-08-01T04:13:55.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.517+0000 7feeea800640 1 -- 172.21.15.3:0/3056491824 wait complete. 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 Processor -- start 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 -- start start 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108180 0x7feee419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4108b30 0x7feee419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 0x7feee41a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feee41182d0 con 0x7feee4108b30 2024-08-01T04:13:55.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feee4118150 con 0x7feee4108180 2024-08-01T04:13:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.518+0000 7feeea800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feee4118450 con 0x7feee410d260 2024-08-01T04:13:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 0x7feee41a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 0x7feee41a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:33988/0 (socket says 172.21.15.3:33988) 2024-08-01T04:13:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3e00640 1 -- 172.21.15.3:0/1548521142 learned_addr learned my addr 172.21.15.3:0/1548521142 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:55.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108180 0x7feee419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3400640 1 -- 172.21.15.3:0/1548521142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 msgr2=0x7feee41a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3400640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 0x7feee41a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3400640 1 -- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4108b30 msgr2=0x7feee419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3400640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4108b30 0x7feee419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.519+0000 7feee3400640 1 -- 172.21.15.3:0/1548521142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee41a2720 con 0x7feee4108180 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.520+0000 7feee3e00640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 0x7feee41a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.520+0000 7feee3400640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108180 0x7feee419e7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feed402a910 tx=0x7feed4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.520+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed403f070 con 0x7feee4108180 2024-08-01T04:13:55.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.520+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed4031c90 con 0x7feee4108180 2024-08-01T04:13:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.520+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4035030 con 0x7feee4108180 2024-08-01T04:13:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.520+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee41a29b0 con 0x7feee4108180 2024-08-01T04:13:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.521+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41ab380 con 0x7feee4108180 2024-08-01T04:13:55.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.522+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee4108560 con 0x7feee4108180 2024-08-01T04:13:55.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.524+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7feed4042070 con 0x7feee4108180 2024-08-01T04:13:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.525+0000 7feee0c00640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feec80a2c00 0x7feec80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.525+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7feed40ef880 con 0x7feee4108180 2024-08-01T04:13:55.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.525+0000 7feedbe00640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feec80a2c00 0x7feec80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:55.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.526+0000 7feedbe00640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feec80a2c00 0x7feec80a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feecc0058c0 tx=0x7feecc005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:55.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.529+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feed40bc8f0 con 0x7feee4108180 2024-08-01T04:13:55.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:55.767+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feee4110440 con 0x7feee4108180 2024-08-01T04:13:55.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:55 smithi003 ceph-mon[32262]: pgmap v286: 97 pgs: 30 creating+peering, 46 unknown, 21 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:55.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:55 smithi003 ceph-mon[32262]: osdmap e84: 12 total, 12 up, 12 in 2024-08-01T04:13:55.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:55 smithi163 ceph-mon[37383]: pgmap v286: 97 pgs: 30 creating+peering, 46 unknown, 21 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:55.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:55 smithi163 ceph-mon[37383]: osdmap e84: 12 total, 12 up, 12 in 2024-08-01T04:13:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:55 smithi160 ceph-mon[37224]: pgmap v286: 97 pgs: 30 creating+peering, 46 unknown, 21 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:55 smithi160 ceph-mon[37224]: osdmap e84: 12 total, 12 up, 12 in 2024-08-01T04:13:56.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.520+0000 7feee0c00640 1 -- 172.21.15.3:0/1548521142 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7feed40bc1f0 con 0x7feee4108180 2024-08-01T04:13:56.523 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-08-01T04:13:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.525+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feec80a2c00 msgr2=0x7feec80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.525+0000 7feeea800640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feec80a2c00 0x7feec80a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feecc0058c0 tx=0x7feecc005850 comp rx=0 tx=0).stop 2024-08-01T04:13:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.525+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108180 msgr2=0x7feee419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:56.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.525+0000 7feeea800640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108180 0x7feee419e7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feed402a910 tx=0x7feed4003b20 comp rx=0 tx=0).stop 2024-08-01T04:13:56.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.526+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 shutdown_connections 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.526+0000 7feeea800640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feec80a2c00 0x7feec80a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.526+0000 7feeea800640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee410d260 0x7feee41a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.526+0000 7feeea800640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feee4108b30 0x7feee419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.526+0000 7feeea800640 1 --2- 172.21.15.3:0/1548521142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feee4108180 0x7feee419e7d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.526+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 >> 172.21.15.3:0/1548521142 conn(0x7feee40778d0 msgr2=0x7feee4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.527+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 shutdown_connections 2024-08-01T04:13:56.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:56.527+0000 7feeea800640 1 -- 172.21.15.3:0/1548521142 wait complete. 2024-08-01T04:13:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:56 smithi163 ceph-mon[37383]: osdmap e85: 12 total, 12 up, 12 in 2024-08-01T04:13:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:56 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1548521142' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-01T04:13:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:56 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-01T04:13:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:56 smithi163 ceph-mon[37383]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-01T04:13:56.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:56 smithi003 ceph-mon[32262]: osdmap e85: 12 total, 12 up, 12 in 2024-08-01T04:13:56.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:56 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1548521142' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-01T04:13:56.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:56 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-01T04:13:56.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:56 smithi003 ceph-mon[32262]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-01T04:13:56.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:56 smithi160 ceph-mon[37224]: osdmap e85: 12 total, 12 up, 12 in 2024-08-01T04:13:56.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:56 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1548521142' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-01T04:13:56.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:56 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-01T04:13:56.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:56 smithi160 ceph-mon[37224]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-01T04:13:57.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:13:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:57 smithi163 ceph-mon[37383]: pgmap v289: 97 pgs: 21 creating+peering, 22 unknown, 54 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:57 smithi163 ceph-mon[37383]: osdmap e86: 12 total, 12 up, 12 in 2024-08-01T04:13:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:57 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-01T04:13:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:57 smithi163 ceph-mon[37383]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-01T04:13:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:57 smithi163 ceph-mon[37383]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-01T04:13:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:57 smithi003 ceph-mon[32262]: pgmap v289: 97 pgs: 21 creating+peering, 22 unknown, 54 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:57 smithi003 ceph-mon[32262]: osdmap e86: 12 total, 12 up, 12 in 2024-08-01T04:13:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:57 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-01T04:13:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:57 smithi003 ceph-mon[32262]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-01T04:13:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:57 smithi003 ceph-mon[32262]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-01T04:13:57.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:57 smithi160 ceph-mon[37224]: pgmap v289: 97 pgs: 21 creating+peering, 22 unknown, 54 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:13:57.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:57 smithi160 ceph-mon[37224]: osdmap e86: 12 total, 12 up, 12 in 2024-08-01T04:13:57.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:57 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-01T04:13:57.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:57 smithi160 ceph-mon[37224]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-01T04:13:57.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:57 smithi160 ceph-mon[37224]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-01T04:13:57.863 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:13:58.152 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:13:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: 2024-08-01T04:13:57.853+0000 7fdfd367d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-01T04:13:58.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:58 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] up:active 2024-08-01T04:13:58.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:58 smithi003 ceph-mon[32262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:13:58.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:58 smithi003 ceph-mon[32262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:13:58.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:58 smithi003 ceph-mon[32262]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-01T04:13:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 -- 172.21.15.3:0/2146699699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100450 msgr2=0x7f7084100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 --2- 172.21.15.3:0/2146699699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100450 0x7f7084100830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7078009ca0 tx=0x7f707802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 -- 172.21.15.3:0/2146699699 shutdown_connections 2024-08-01T04:13:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 --2- 172.21.15.3:0/2146699699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f708410e000 0x7f70841103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 --2- 172.21.15.3:0/2146699699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7084100e00 0x7f708410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 --2- 172.21.15.3:0/2146699699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100450 0x7f7084100830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.546+0000 7f708ac00640 1 -- 172.21.15.3:0/2146699699 >> 172.21.15.3:0/2146699699 conn(0x7f70840fc340 msgr2=0x7f70840fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.547+0000 7f708ac00640 1 -- 172.21.15.3:0/2146699699 shutdown_connections 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.547+0000 7f708ac00640 1 -- 172.21.15.3:0/2146699699 wait complete. 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 Processor -- start 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 -- start start 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7084100450 0x7f708419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100e00 0x7f708419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f708410e000 0x7f708419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7084113d50 con 0x7f7084100e00 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7084113bd0 con 0x7f708410e000 2024-08-01T04:13:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.548+0000 7f708ac00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7084113ed0 con 0x7f7084100450 2024-08-01T04:13:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f7088e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100e00 0x7f708419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f7088e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100e00 0x7f708419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33204/0 (socket says 172.21.15.3:33204) 2024-08-01T04:13:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f7088e00640 1 -- 172.21.15.3:0/970791216 learned_addr learned my addr 172.21.15.3:0/970791216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f708410e000 0x7f708419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f7089800640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7084100450 0x7f708419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 -- 172.21.15.3:0/970791216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7084100450 msgr2=0x7f708419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7084100450 0x7f708419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 -- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100e00 msgr2=0x7f708419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100e00 0x7f708419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 -- 172.21.15.3:0/970791216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708419f240 con 0x7f708410e000 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.549+0000 7f708a200640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f708410e000 0x7f708419eb60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7084102630 tx=0x7f707c00dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.550+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c017070 con 0x7f708410e000 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.550+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707c0047a0 con 0x7f708410e000 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.550+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c009ce0 con 0x7f708410e000 2024-08-01T04:13:58.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.550+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708419f530 con 0x7f708410e000 2024-08-01T04:13:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.550+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70841a6e10 con 0x7f708410e000 2024-08-01T04:13:58.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.551+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7084100830 con 0x7f708410e000 2024-08-01T04:13:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.553+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f707c004950 con 0x7f708410e000 2024-08-01T04:13:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.554+0000 7f7082000640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f706c0a2c00 0x7f706c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:13:58.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.555+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f707c0ce700 con 0x7f708410e000 2024-08-01T04:13:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.555+0000 7f7089800640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f706c0a2c00 0x7f706c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:13:58.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.556+0000 7f7089800640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f706c0a2c00 0x7f706c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7078006010 tx=0x7f7078009820 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:13:58.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.559+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f707c09b0c0 con 0x7f708410e000 2024-08-01T04:13:58.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.758+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70841091f0 con 0x7f708410e000 2024-08-01T04:13:58.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.760+0000 7f7082000640 1 -- 172.21.15.3:0/970791216 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22186 (secure 0 0 0) 0x7f707c09f3c0 con 0x7f708410e000 2024-08-01T04:13:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:13:58.763 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":86,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:13:55.770103+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:13:58.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f706c0a2c00 msgr2=0x7f706c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:58.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f706c0a2c00 0x7f706c0a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7078006010 tx=0x7f7078009820 comp rx=0 tx=0).stop 2024-08-01T04:13:58.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f708410e000 msgr2=0x7f708419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f708410e000 0x7f708419eb60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7084102630 tx=0x7f707c00dae0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 shutdown_connections 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f706c0a2c00 0x7f706c0a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f708410e000 0x7f708419eb60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7084100e00 0x7f708419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 --2- 172.21.15.3:0/970791216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7084100450 0x7f708419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.763+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 >> 172.21.15.3:0/970791216 conn(0x7f70840fc340 msgr2=0x7f70840fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.764+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 shutdown_connections 2024-08-01T04:13:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:13:58.764+0000 7f708ac00640 1 -- 172.21.15.3:0/970791216 wait complete. 2024-08-01T04:13:58.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:58 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] up:active 2024-08-01T04:13:58.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:58 smithi163 ceph-mon[37383]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:13:58.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:58 smithi163 ceph-mon[37383]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:13:58.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:58 smithi163 ceph-mon[37383]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-01T04:13:58.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:58 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] up:active 2024-08-01T04:13:58.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:58 smithi160 ceph-mon[37224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:13:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:58 smithi160 ceph-mon[37224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:13:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:58 smithi160 ceph-mon[37224]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-01T04:13:59.582 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:13:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:59 smithi163 ceph-mon[37383]: pgmap v291: 97 pgs: 21 creating+peering, 6 unknown, 70 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 869 B/s wr, 2 op/s 2024-08-01T04:13:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:59 smithi163 ceph-mon[37383]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-01T04:13:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:59 smithi163 ceph-mon[37383]: Cluster is now healthy 2024-08-01T04:13:59.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:13:59 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/970791216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:13:59.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:59 smithi003 ceph-mon[32262]: pgmap v291: 97 pgs: 21 creating+peering, 6 unknown, 70 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 869 B/s wr, 2 op/s 2024-08-01T04:13:59.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:59 smithi003 ceph-mon[32262]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-01T04:13:59.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:59 smithi003 ceph-mon[32262]: Cluster is now healthy 2024-08-01T04:13:59.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:13:59 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/970791216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:13:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:59 smithi160 ceph-mon[37224]: pgmap v291: 97 pgs: 21 creating+peering, 6 unknown, 70 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 869 B/s wr, 2 op/s 2024-08-01T04:13:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:59 smithi160 ceph-mon[37224]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-01T04:13:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:59 smithi160 ceph-mon[37224]: Cluster is now healthy 2024-08-01T04:13:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:13:59 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/970791216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:00.036 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 -- 172.21.15.3:0/2150438670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 msgr2=0x7f6a941107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 --2- 172.21.15.3:0/2150438670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 0x7f6a941107b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009c10 tx=0x7f6a8c02a6f0 comp rx=0 tx=0).stop 2024-08-01T04:14:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 -- 172.21.15.3:0/2150438670 shutdown_connections 2024-08-01T04:14:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 --2- 172.21.15.3:0/2150438670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 0x7f6a941107b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 --2- 172.21.15.3:0/2150438670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a941038e0 0x7f6a94103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 --2- 172.21.15.3:0/2150438670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a94109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 -- 172.21.15.3:0/2150438670 >> 172.21.15.3:0/2150438670 conn(0x7f6a940ff5a0 msgr2=0x7f6a941019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 -- 172.21.15.3:0/2150438670 shutdown_connections 2024-08-01T04:14:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.904+0000 7f6a98c00640 1 -- 172.21.15.3:0/2150438670 wait complete. 2024-08-01T04:14:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.905+0000 7f6a98c00640 1 Processor -- start 2024-08-01T04:14:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.905+0000 7f6a98c00640 1 -- start start 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a98c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a941038e0 0x7f6a9419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a98c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 0x7f6a9419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a98c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a9419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a98c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94113ce0 con 0x7f6a941098e0 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a98c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94113b60 con 0x7f6a941038e0 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a98c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94113e60 con 0x7f6a94104280 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a93e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a9419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a93e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a9419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33230/0 (socket says 172.21.15.3:33230) 2024-08-01T04:14:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a93e00640 1 -- 172.21.15.3:0/3424802832 learned_addr learned my addr 172.21.15.3:0/3424802832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a92a00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 0x7f6a9419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.906+0000 7f6a93400640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a941038e0 0x7f6a9419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a93e00640 1 -- 172.21.15.3:0/3424802832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 msgr2=0x7f6a9419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a93e00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 0x7f6a9419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a93e00640 1 -- 172.21.15.3:0/3424802832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a941038e0 msgr2=0x7f6a9419a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a93e00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a941038e0 0x7f6a9419a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a93e00640 1 -- 172.21.15.3:0/3424802832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9419f0f0 con 0x7f6a941098e0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a93e00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a9419ea10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02ac10 tx=0x7f6a8c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c041020 con 0x7f6a941098e0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.907+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c003e50 con 0x7f6a941098e0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.908+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9419f380 con 0x7f6a941098e0 2024-08-01T04:14:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.908+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c036040 con 0x7f6a941098e0 2024-08-01T04:14:00.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.908+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a6cc0 con 0x7f6a941098e0 2024-08-01T04:14:00.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.909+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a94105020 con 0x7f6a941098e0 2024-08-01T04:14:00.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.911+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6a8c044020 con 0x7f6a941098e0 2024-08-01T04:14:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.913+0000 7f6a8be00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6a780a2b80 0x7f6a780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.913+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6a8c0eff70 con 0x7f6a941098e0 2024-08-01T04:14:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.913+0000 7f6a93400640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6a780a2b80 0x7f6a780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:00.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.914+0000 7f6a93400640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6a780a2b80 0x7f6a780a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a84005fd0 tx=0x7f6a84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:00.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:00.916+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6a8c04b020 con 0x7f6a941098e0 2024-08-01T04:14:01.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.114+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a9410bed0 con 0x7f6a941098e0 2024-08-01T04:14:01.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.117+0000 7f6a8be00640 1 -- 172.21.15.3:0/3424802832 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22186 (secure 0 0 0) 0x7f6a8c0bcf80 con 0x7f6a941098e0 2024-08-01T04:14:01.120 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:01.120 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":86,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:13:55.770103+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:14:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.120+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6a780a2b80 msgr2=0x7f6a780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.120+0000 7f6a98c00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6a780a2b80 0x7f6a780a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a84005fd0 tx=0x7f6a84009260 comp rx=0 tx=0).stop 2024-08-01T04:14:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.120+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 msgr2=0x7f6a9419ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.120+0000 7f6a98c00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a9419ea10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c02ac10 tx=0x7f6a8c0305d0 comp rx=0 tx=0).stop 2024-08-01T04:14:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 shutdown_connections 2024-08-01T04:14:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6a780a2b80 0x7f6a780a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a941098e0 0x7f6a9419ea10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a94104280 0x7f6a9419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 --2- 172.21.15.3:0/3424802832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a941038e0 0x7f6a9419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 >> 172.21.15.3:0/3424802832 conn(0x7f6a940ff5a0 msgr2=0x7f6a94100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:01.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 shutdown_connections 2024-08-01T04:14:01.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:01.121+0000 7f6a98c00640 1 -- 172.21.15.3:0/3424802832 wait complete. 2024-08-01T04:14:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:01 smithi163 ceph-mon[37383]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-08-01T04:14:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:01 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3424802832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:01.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:01 smithi003 ceph-mon[32262]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-08-01T04:14:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:01 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3424802832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:01.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:01 smithi160 ceph-mon[37224]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-08-01T04:14:01.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:01 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3424802832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:01.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-01T04:14:02.440 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 -- 172.21.15.3:0/2610448554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c104480 msgr2=0x7f030c1109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 --2- 172.21.15.3:0/2610448554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c104480 0x7f030c1109b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0304009930 tx=0x7f030402a420 comp rx=0 tx=0).stop 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 -- 172.21.15.3:0/2610448554 shutdown_connections 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 --2- 172.21.15.3:0/2610448554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c104480 0x7f030c1109b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 --2- 172.21.15.3:0/2610448554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c103ae0 0x7f030c103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 --2- 172.21.15.3:0/2610448554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c109b70 0x7f030c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 -- 172.21.15.3:0/2610448554 >> 172.21.15.3:0/2610448554 conn(0x7f030c0ff780 msgr2=0x7f030c101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.121+0000 7f0311400640 1 -- 172.21.15.3:0/2610448554 shutdown_connections 2024-08-01T04:14:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.122+0000 7f0311400640 1 -- 172.21.15.3:0/2610448554 wait complete. 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 Processor -- start 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 -- start start 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c103ae0 0x7f030c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 0x7f030c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c109b70 0x7f030c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f030c113f30 con 0x7f030c109b70 2024-08-01T04:14:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f030c113db0 con 0x7f030c104480 2024-08-01T04:14:03.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.123+0000 7f0311400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f030c1140b0 con 0x7f030c103ae0 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f030b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c109b70 0x7f030c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 0x7f030c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f030a800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c103ae0 0x7f030c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 0x7f030c19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:57568/0 (socket says 172.21.15.3:57568) 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 -- 172.21.15.3:0/2544975711 learned_addr learned my addr 172.21.15.3:0/2544975711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 -- 172.21.15.3:0/2544975711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c103ae0 msgr2=0x7f030c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c103ae0 0x7f030c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 -- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c109b70 msgr2=0x7f030c19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c109b70 0x7f030c19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f0309e00640 1 -- 172.21.15.3:0/2544975711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030c19f320 con 0x7f030c104480 2024-08-01T04:14:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.124+0000 7f030a800640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c103ae0 0x7f030c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.125+0000 7f0309e00640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 0x7f030c19a8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02f400ebc0 tx=0x7f02f400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.125+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4010630 con 0x7f030c104480 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.125+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030c19f610 con 0x7f030c104480 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.125+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f4010cb0 con 0x7f030c104480 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.125+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4018bd0 con 0x7f030c104480 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.125+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030c1a6ef0 con 0x7f030c104480 2024-08-01T04:14:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.126+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030c103f40 con 0x7f030c104480 2024-08-01T04:14:03.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.128+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f02f40107d0 con 0x7f030c104480 2024-08-01T04:14:03.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.130+0000 7f0303400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02f00a2b80 0x7f02f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:03.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.130+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f02f40cf780 con 0x7f030c104480 2024-08-01T04:14:03.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.130+0000 7f030a800640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02f00a2b80 0x7f02f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:03.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.131+0000 7f030a800640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02f00a2b80 0x7f02f00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009f40 tx=0x7f02fc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:03.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.131+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f02f409c140 con 0x7f030c104480 2024-08-01T04:14:03.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.368+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f030c1a7210 con 0x7f030c104480 2024-08-01T04:14:03.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.540+0000 7f0303400640 1 -- 172.21.15.3:0/2544975711 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f02f40a0440 con 0x7f030c104480 2024-08-01T04:14:03.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02f00a2b80 msgr2=0x7f02f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:03.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02f00a2b80 0x7f02f00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009f40 tx=0x7f02fc005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 msgr2=0x7f030c19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 0x7f030c19a8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02f400ebc0 tx=0x7f02f400c6a0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 shutdown_connections 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02f00a2b80 0x7f02f00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f030c109b70 0x7f030c19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030c104480 0x7f030c19a8b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 --2- 172.21.15.3:0/2544975711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f030c103ae0 0x7f030c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:03.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.544+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 >> 172.21.15.3:0/2544975711 conn(0x7f030c0ff780 msgr2=0x7f030c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.545+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 shutdown_connections 2024-08-01T04:14:03.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:03.545+0000 7f0311400640 1 -- 172.21.15.3:0/2544975711 wait complete. 2024-08-01T04:14:03.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:03 smithi003 ceph-mon[32262]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-08-01T04:14:03.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:03 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2544975711' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-01T04:14:03.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:03 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-01T04:14:03.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:03 smithi160 ceph-mon[37224]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-08-01T04:14:03.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:03 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2544975711' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-01T04:14:03.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:03 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-01T04:14:04.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:03 smithi163 ceph-mon[37383]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-08-01T04:14:04.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:03 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2544975711' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-01T04:14:04.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:03 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-01T04:14:04.461 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-08-01T04:14:04.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-01T04:14:04.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:04 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-01T04:14:04.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:04 smithi003 ceph-mon[32262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:14:04.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:04 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-01T04:14:04.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:04 smithi160 ceph-mon[37224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:14:04.936 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:05.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:04 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-01T04:14:05.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:04 smithi163 ceph-mon[37383]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:14:05.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 -- 172.21.15.3:0/2744776472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1016f0 msgr2=0x7feddc101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 --2- 172.21.15.3:0/2744776472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1016f0 0x7feddc101b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009930 tx=0x7fedc402a420 comp rx=0 tx=0).stop 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 -- 172.21.15.3:0/2744776472 shutdown_connections 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 --2- 172.21.15.3:0/2744776472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feddc102090 0x7feddc10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 --2- 172.21.15.3:0/2744776472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1016f0 0x7feddc101b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 --2- 172.21.15.3:0/2744776472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc1076f0 0x7feddc107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.597+0000 7feddbe00640 1 -- 172.21.15.3:0/2744776472 >> 172.21.15.3:0/2744776472 conn(0x7feddc0fd470 msgr2=0x7feddc0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:05.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.598+0000 7feddbe00640 1 -- 172.21.15.3:0/2744776472 shutdown_connections 2024-08-01T04:14:05.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.598+0000 7feddbe00640 1 -- 172.21.15.3:0/2744776472 wait complete. 2024-08-01T04:14:05.601 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:05 smithi003 ceph-mon[32262]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-01T04:14:05.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 Processor -- start 2024-08-01T04:14:05.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 -- start start 2024-08-01T04:14:05.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feddc1016f0 0x7feddc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc102090 0x7feddc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 0x7feddc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feddc111b40 con 0x7feddc1076f0 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1119c0 con 0x7feddc1016f0 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.599+0000 7feddbe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feddc111cc0 con 0x7feddc102090 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 0x7feddc19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 0x7feddc19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58088/0 (socket says 172.21.15.3:58088) 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 -- 172.21.15.3:0/2554795472 learned_addr learned my addr 172.21.15.3:0/2554795472 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7fedd2000640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc102090 0x7feddc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:05.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddaa00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feddc1016f0 0x7feddc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 -- 172.21.15.3:0/2554795472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc102090 msgr2=0x7feddc1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc102090 0x7feddc1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 -- 172.21.15.3:0/2554795472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feddc1016f0 msgr2=0x7feddc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feddc1016f0 0x7feddc198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7feddb400640 1 -- 172.21.15.3:0/2554795472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc19d030 con 0x7feddc1076f0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.600+0000 7fedd2000640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc102090 0x7feddc1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.601+0000 7feddb400640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 0x7feddc19c950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fedd400efd0 tx=0x7fedd400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.601+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4003b70 con 0x7feddc1076f0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.601+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd4003d10 con 0x7feddc1076f0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.601+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40109f0 con 0x7feddc1076f0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.601+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc19d320 con 0x7feddc1076f0 2024-08-01T04:14:05.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.601+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a4c00 con 0x7feddc1076f0 2024-08-01T04:14:05.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.603+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc101b50 con 0x7feddc1076f0 2024-08-01T04:14:05.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.606+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fedd4010b90 con 0x7feddc1076f0 2024-08-01T04:14:05.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.608+0000 7fedd3e00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fedc00a2b80 0x7fedc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:05.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.608+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fedd40cf020 con 0x7feddc1076f0 2024-08-01T04:14:05.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.609+0000 7feddaa00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fedc00a2b80 0x7fedc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:05.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.609+0000 7feddaa00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fedc00a2b80 0x7fedc00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009b50 tx=0x7fedcc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:05.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.613+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fedd409c030 con 0x7feddc1076f0 2024-08-01T04:14:05.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:05.847+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7feddc1096d0 con 0x7feddc1076f0 2024-08-01T04:14:05.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:05 smithi160 ceph-mon[37224]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-01T04:14:06.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:05 smithi163 ceph-mon[37383]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-08-01T04:14:06.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.572+0000 7fedd3e00640 1 -- 172.21.15.3:0/2554795472 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fedd409b930 con 0x7feddc1076f0 2024-08-01T04:14:06.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fedc00a2b80 msgr2=0x7fedc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:06.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fedc00a2b80 0x7fedc00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009b50 tx=0x7fedcc005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:06.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 msgr2=0x7feddc19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 0x7feddc19c950 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fedd400efd0 tx=0x7fedd400c6a0 comp rx=0 tx=0).stop 2024-08-01T04:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 shutdown_connections 2024-08-01T04:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fedc00a2b80 0x7fedc00a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feddc1076f0 0x7feddc19c950 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feddc102090 0x7feddc1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:06.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 --2- 172.21.15.3:0/2554795472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feddc1016f0 0x7feddc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.576+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 >> 172.21.15.3:0/2554795472 conn(0x7feddc0fd470 msgr2=0x7feddc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.577+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 shutdown_connections 2024-08-01T04:14:06.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:06.577+0000 7feddbe00640 1 -- 172.21.15.3:0/2554795472 wait complete. 2024-08-01T04:14:06.816 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:06 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2554795472' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-01T04:14:06.925 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:06 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2554795472' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-01T04:14:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:06 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2554795472' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-01T04:14:07.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-01T04:14:07.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-01T04:14:07.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-01T04:14:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2554795472' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-01T04:14:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-01T04:14:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:14:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-08-01T04:14:07.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:07 smithi003 ceph-mon[32262]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2554795472' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-08-01T04:14:07.856 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-mon[37224]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-08-01T04:14:07.976 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2554795472' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-08-01T04:14:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:07 smithi163 ceph-mon[37383]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-08-01T04:14:08.152 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:14:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: 2024-08-01T04:14:07.853+0000 7fdfd367d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] up:active 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: Cluster is now healthy 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2024-08-01T04:14:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:08 smithi003 ceph-mon[32262]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-01T04:14:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 -- 172.21.15.3:0/1433614913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 msgr2=0x7feb68114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 --2- 172.21.15.3:0/1433614913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 0x7feb68114de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb60009ca0 tx=0x7feb6002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 -- 172.21.15.3:0/1433614913 shutdown_connections 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 --2- 172.21.15.3:0/1433614913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 0x7feb68114de0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 --2- 172.21.15.3:0/1433614913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 0x7feb6810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 --2- 172.21.15.3:0/1433614913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb68108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 -- 172.21.15.3:0/1433614913 >> 172.21.15.3:0/1433614913 conn(0x7feb680778d0 msgr2=0x7feb68079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 -- 172.21.15.3:0/1433614913 shutdown_connections 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.696+0000 7feb70600640 1 -- 172.21.15.3:0/1433614913 wait complete. 2024-08-01T04:14:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.697+0000 7feb70600640 1 Processor -- start 2024-08-01T04:14:08.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 -- start start 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb6819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 0x7feb6819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 0x7feb681a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb681182e0 con 0x7feb68108b30 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feb68118160 con 0x7feb68108180 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb70600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feb68118460 con 0x7feb6810d260 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.698+0000 7feb6dc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb6819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb6819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:57604/0 (socket says 172.21.15.3:57604) 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 -- 172.21.15.3:0/1422697224 learned_addr learned my addr 172.21.15.3:0/1422697224 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:08.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6d200640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 0x7feb6819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6e600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 0x7feb681a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 -- 172.21.15.3:0/1422697224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 msgr2=0x7feb681a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 0x7feb681a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 -- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 msgr2=0x7feb6819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 0x7feb6819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6dc00640 1 -- 172.21.15.3:0/1422697224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb681a36d0 con 0x7feb68108180 2024-08-01T04:14:08.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.699+0000 7feb6d200640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 0x7feb6819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.700+0000 7feb6dc00640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb6819e720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00e9f0 tx=0x7feb5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.700+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c00cd80 con 0x7feb68108180 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.700+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb5c005500 con 0x7feb68108180 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.700+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c010690 con 0x7feb68108180 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.700+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb681a39c0 con 0x7feb68108180 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.700+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb681ab2a0 con 0x7feb68108180 2024-08-01T04:14:08.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.701+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb68108560 con 0x7feb68108180 2024-08-01T04:14:08.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.703+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7feb5c0046c0 con 0x7feb68108180 2024-08-01T04:14:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.704+0000 7feb66000640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feb500a2c00 0x7feb500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.704+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7feb5c0cf310 con 0x7feb68108180 2024-08-01T04:14:08.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.705+0000 7feb6d200640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feb500a2c00 0x7feb500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:08.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.705+0000 7feb6d200640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feb500a2c00 0x7feb500a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb68107c80 tx=0x7feb58005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:08.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.708+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feb5c09c320 con 0x7feb68108180 2024-08-01T04:14:08.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] up:active 2024-08-01T04:14:08.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-01T04:14:08.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-01T04:14:08.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: Cluster is now healthy 2024-08-01T04:14:08.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-08-01T04:14:08.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2024-08-01T04:14:08.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:08 smithi160 ceph-mon[37224]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-01T04:14:08.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:08.942+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7feb681ab470 con 0x7feb68108180 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] up:active 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: Cluster is now healthy 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2024-08-01T04:14:09.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:08 smithi163 ceph-mon[37383]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-01T04:14:09.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.614+0000 7feb66000640 1 -- 172.21.15.3:0/1422697224 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7feb5c086070 con 0x7feb68108180 2024-08-01T04:14:09.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feb500a2c00 msgr2=0x7feb500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:09.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feb500a2c00 0x7feb500a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb68107c80 tx=0x7feb58005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:09.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 msgr2=0x7feb6819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:09.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb6819e720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00e9f0 tx=0x7feb5c00edf0 comp rx=0 tx=0).stop 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 shutdown_connections 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feb500a2c00 0x7feb500a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb6810d260 0x7feb681a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb68108b30 0x7feb6819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 --2- 172.21.15.3:0/1422697224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb68108180 0x7feb6819e720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.619+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 >> 172.21.15.3:0/1422697224 conn(0x7feb680778d0 msgr2=0x7feb68079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.620+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 shutdown_connections 2024-08-01T04:14:09.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:09.620+0000 7feb70600640 1 -- 172.21.15.3:0/1422697224 wait complete. 2024-08-01T04:14:09.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:09 smithi160 ceph-mon[37224]: pgmap v296: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-08-01T04:14:09.905 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:09 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] up:active 2024-08-01T04:14:09.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:09 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2024-08-01T04:14:09.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:09 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1422697224' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-01T04:14:09.906 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:09 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-01T04:14:10.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:09 smithi003 ceph-mon[32262]: pgmap v296: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-08-01T04:14:10.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:09 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] up:active 2024-08-01T04:14:10.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:09 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2024-08-01T04:14:10.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:09 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1422697224' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-01T04:14:10.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:09 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-01T04:14:10.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:09 smithi163 ceph-mon[37383]: pgmap v296: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-08-01T04:14:10.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:09 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] up:active 2024-08-01T04:14:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:09 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2024-08-01T04:14:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:09 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1422697224' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-01T04:14:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:09 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-01T04:14:10.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-01T04:14:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:10 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-01T04:14:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:10 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2024-08-01T04:14:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:10 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:14:10.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:10 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-01T04:14:10.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:10 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2024-08-01T04:14:10.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:10 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:14:10.950 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:11.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:10 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-01T04:14:11.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:10 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 9 up:standby 2024-08-01T04:14:11.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:10 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:14:11.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 -- 172.21.15.3:0/3684392981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 msgr2=0x7feafc076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:11.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 --2- 172.21.15.3:0/3684392981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc076830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009ca0 tx=0x7feaf002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:14:11.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 -- 172.21.15.3:0/3684392981 shutdown_connections 2024-08-01T04:14:11.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 --2- 172.21.15.3:0/3684392981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feafc1125a0 0x7feafc114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 --2- 172.21.15.3:0/3684392981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 0x7feafc112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 --2- 172.21.15.3:0/3684392981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc076830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 -- 172.21.15.3:0/3684392981 >> 172.21.15.3:0/3684392981 conn(0x7feafc0ff600 msgr2=0x7feafc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 -- 172.21.15.3:0/3684392981 shutdown_connections 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.604+0000 7feb02600640 1 -- 172.21.15.3:0/3684392981 wait complete. 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.605+0000 7feb02600640 1 Processor -- start 2024-08-01T04:14:11.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.605+0000 7feb02600640 1 -- start start 2024-08-01T04:14:11.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb02600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb02600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 0x7feafc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb02600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feafc1125a0 0x7feafc1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb02600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feafc118160 con 0x7feafc076450 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb02600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feafc117fe0 con 0x7feafc076d70 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb02600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1182e0 con 0x7feafc1125a0 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb01200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb01200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58128/0 (socket says 172.21.15.3:58128) 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb01200640 1 -- 172.21.15.3:0/884631796 learned_addr learned my addr 172.21.15.3:0/884631796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feafbe00640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 0x7feafc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:11.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.606+0000 7feb01c00640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feafc1125a0 0x7feafc1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feb01200640 1 -- 172.21.15.3:0/884631796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feafc1125a0 msgr2=0x7feafc1a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feb01200640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feafc1125a0 0x7feafc1a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feb01200640 1 -- 172.21.15.3:0/884631796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 msgr2=0x7feafc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feb01200640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 0x7feafc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feb01200640 1 -- 172.21.15.3:0/884631796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc1a35b0 con 0x7feafc076450 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feafbe00640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 0x7feafc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.607+0000 7feb01200640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc19e600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005e30 tx=0x7feaf0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.608+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00316d0 con 0x7feafc076450 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.608+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0031870 con 0x7feafc076450 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.608+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002fb70 con 0x7feafc076450 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.608+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc1a3840 con 0x7feafc076450 2024-08-01T04:14:11.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.608+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc1ab180 con 0x7feafc076450 2024-08-01T04:14:11.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.609+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feafc077b90 con 0x7feafc076450 2024-08-01T04:14:11.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.611+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7feaf002fd50 con 0x7feafc076450 2024-08-01T04:14:11.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.613+0000 7feaf9600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feae40a2c00 0x7feae40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.613+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7feaf00f02c0 con 0x7feafc076450 2024-08-01T04:14:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.613+0000 7feafbe00640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feae40a2c00 0x7feae40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:11.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.614+0000 7feafbe00640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feae40a2c00 0x7feae40a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feae80095f0 tx=0x7feae8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:11.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.618+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feaf00bd360 con 0x7feafc076450 2024-08-01T04:14:11.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:11 smithi003 ceph-mon[32262]: pgmap v297: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-08-01T04:14:11.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:11.858+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7feafc10d730 con 0x7feafc076450 2024-08-01T04:14:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:11 smithi160 ceph-mon[37224]: pgmap v297: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-08-01T04:14:12.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:11 smithi163 ceph-mon[37383]: pgmap v297: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-08-01T04:14:12.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.641+0000 7feaf9600640 1 -- 172.21.15.3:0/884631796 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7feaf00bcc60 con 0x7feafc076450 2024-08-01T04:14:12.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feae40a2c00 msgr2=0x7feae40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:12.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feae40a2c00 0x7feae40a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feae80095f0 tx=0x7feae8005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:12.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 msgr2=0x7feafc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:12.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc19e600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005e30 tx=0x7feaf0003f90 comp rx=0 tx=0).stop 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 shutdown_connections 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7feae40a2c00 0x7feae40a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feafc1125a0 0x7feafc1a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.645+0000 7feb02600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feafc076d70 0x7feafc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.646+0000 7feb02600640 1 --2- 172.21.15.3:0/884631796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feafc076450 0x7feafc19e600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.646+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 >> 172.21.15.3:0/884631796 conn(0x7feafc0ff600 msgr2=0x7feafc1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.646+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 shutdown_connections 2024-08-01T04:14:12.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:12.646+0000 7feb02600640 1 -- 172.21.15.3:0/884631796 wait complete. 2024-08-01T04:14:12.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:12 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/884631796' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-01T04:14:13.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:12 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/884631796' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-01T04:14:13.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:12 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/884631796' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-01T04:14:13.499 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-08-01T04:14:13.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-01T04:14:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:13 smithi003 ceph-mon[32262]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-08-01T04:14:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:13 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/884631796' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-01T04:14:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:13 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:14:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:13 smithi160 ceph-mon[37224]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-08-01T04:14:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:13 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/884631796' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-01T04:14:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:13 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:14:13.946 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:13 smithi163 ceph-mon[37383]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-08-01T04:14:13.946 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:13 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/884631796' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-01T04:14:13.946 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:13 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:14:13.984 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:14.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.781+0000 7f9d21600640 1 -- 172.21.15.3:0/3209153767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c106690 msgr2=0x7f9d1c106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:14.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.781+0000 7f9d21600640 1 --2- 172.21.15.3:0/3209153767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c106690 0x7f9d1c106a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009890 tx=0x7f9d0c02a360 comp rx=0 tx=0).stop 2024-08-01T04:14:14.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 -- 172.21.15.3:0/3209153767 shutdown_connections 2024-08-01T04:14:14.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 --2- 172.21.15.3:0/3209153767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c100fc0 0x7f9d1c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:14.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 --2- 172.21.15.3:0/3209153767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 --2- 172.21.15.3:0/3209153767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c106690 0x7f9d1c106a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 -- 172.21.15.3:0/3209153767 >> 172.21.15.3:0/3209153767 conn(0x7f9d1c0fc340 msgr2=0x7f9d1c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 -- 172.21.15.3:0/3209153767 shutdown_connections 2024-08-01T04:14:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.782+0000 7f9d21600640 1 -- 172.21.15.3:0/3209153767 wait complete. 2024-08-01T04:14:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.783+0000 7f9d21600640 1 Processor -- start 2024-08-01T04:14:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 -- start start 2024-08-01T04:14:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c100fc0 0x7f9d1c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c106690 0x7f9d1c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c1109f0 con 0x7f9d1c106690 2024-08-01T04:14:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c110870 con 0x7f9d1c100600 2024-08-01T04:14:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.784+0000 7f9d21600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c110b70 con 0x7f9d1c100fc0 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:35906/0 (socket says 172.21.15.3:35906) 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 -- 172.21.15.3:0/2901616327 learned_addr learned my addr 172.21.15.3:0/2901616327 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d20c00640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c106690 0x7f9d1c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d13400640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c100fc0 0x7f9d1c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 -- 172.21.15.3:0/2901616327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c100fc0 msgr2=0x7f9d1c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c100fc0 0x7f9d1c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 -- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c106690 msgr2=0x7f9d1c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:14.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c106690 0x7f9d1c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 -- 172.21.15.3:0/2901616327 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c19f130 con 0x7f9d1c100600 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d20c00640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c106690 0x7f9d1c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d13400640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c100fc0 0x7f9d1c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.785+0000 7f9d1be00640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c19a180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c035040 tx=0x7f9d0c033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.786+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c037040 con 0x7f9d1c100600 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.786+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d0c040070 con 0x7f9d1c100600 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.786+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c19f3c0 con 0x7f9d1c100600 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.786+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c003970 con 0x7f9d1c100600 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.786+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a6d00 con 0x7f9d1c100600 2024-08-01T04:14:14.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.787+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c100a80 con 0x7f9d1c100600 2024-08-01T04:14:14.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.789+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9d0c0306a0 con 0x7f9d1c100600 2024-08-01T04:14:14.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.790+0000 7f9d19600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9d000a2c00 0x7f9d000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:14.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.791+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9d0c0eef90 con 0x7f9d1c100600 2024-08-01T04:14:14.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.791+0000 7f9d13400640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9d000a2c00 0x7f9d000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:14.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.791+0000 7f9d13400640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9d000a2c00 0x7f9d000a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0fe6e0 tx=0x7f9d04005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:14.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:14.795+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9d0c0bbfa0 con 0x7f9d1c100600 2024-08-01T04:14:15.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.015+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9d1c06a3d0 con 0x7f9d000a2c00 2024-08-01T04:14:15.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.053+0000 7f9d19600640 1 -- 172.21.15.3:0/2901616327 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d1c06a3d0 con 0x7f9d000a2c00 2024-08-01T04:14:15.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.055+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9d000a2c00 msgr2=0x7f9d000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:15.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.055+0000 7f9d21600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9d000a2c00 0x7f9d000a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0fe6e0 tx=0x7f9d04005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:15.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 msgr2=0x7f9d1c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:15.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c19a180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c035040 tx=0x7f9d0c033b60 comp rx=0 tx=0).stop 2024-08-01T04:14:15.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 shutdown_connections 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f9d000a2c00 0x7f9d000a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9d1c106690 0x7f9d1c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d1c100fc0 0x7f9d1c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 --2- 172.21.15.3:0/2901616327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d1c100600 0x7f9d1c19a180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.056+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 >> 172.21.15.3:0/2901616327 conn(0x7f9d1c0fc340 msgr2=0x7f9d1c069cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.057+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 shutdown_connections 2024-08-01T04:14:15.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:15.057+0000 7f9d21600640 1 -- 172.21.15.3:0/2901616327 wait complete. 2024-08-01T04:14:15.299 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:14:15.025+0000 7f3a71e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:14:15.299 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:14:15.025+0000 7f3a71e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:14:15.299 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:14:15.025+0000 7f3a71e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:14:15.299 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:14:15.025+0000 7f3a71e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:14:15.299 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:14:15.025+0000 7f3a71e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:14:15.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-mon[32262]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-08-01T04:14:15.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:15 smithi003 ceph-mon[32262]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:15.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-01T04:14:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:15 smithi163 ceph-mon[37383]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-08-01T04:14:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:15 smithi163 ceph-mon[37383]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:16.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:15 smithi160 ceph-mon[37224]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-08-01T04:14:16.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:15 smithi160 ceph-mon[37224]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:16.467 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 -- 172.21.15.3:0/4060376816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b0100fd0 msgr2=0x7fa9b01054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:17.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 --2- 172.21.15.3:0/4060376816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b0100fd0 0x7fa9b01054a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009900 tx=0x7fa99c02a470 comp rx=0 tx=0).stop 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 -- 172.21.15.3:0/4060376816 shutdown_connections 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 --2- 172.21.15.3:0/4060376816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b0105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 --2- 172.21.15.3:0/4060376816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b0100fd0 0x7fa9b01054a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 --2- 172.21.15.3:0/4060376816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9b01006b0 0x7fa9b0100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 -- 172.21.15.3:0/4060376816 >> 172.21.15.3:0/4060376816 conn(0x7fa9b00fc340 msgr2=0x7fa9b00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 -- 172.21.15.3:0/4060376816 shutdown_connections 2024-08-01T04:14:17.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.152+0000 7fa9afe00640 1 -- 172.21.15.3:0/4060376816 wait complete. 2024-08-01T04:14:17.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.153+0000 7fa9afe00640 1 Processor -- start 2024-08-01T04:14:17.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 -- start start 2024-08-01T04:14:17.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b01006b0 0x7fa9b019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:17.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9b0100fd0 0x7fa9b019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:17.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b01a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:17.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0118110 con 0x7fa9b01006b0 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0117f90 con 0x7fa9b0100fd0 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9afe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0118290 con 0x7fa9b0105ac0 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9af400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b01a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9aea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b01006b0 0x7fa9b019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9ae000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9b0100fd0 0x7fa9b019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9af400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b01a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:35590/0 (socket says 172.21.15.3:35590) 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.154+0000 7fa9af400640 1 -- 172.21.15.3:0/1967387761 learned_addr learned my addr 172.21.15.3:0/1967387761 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.155+0000 7fa9af400640 1 -- 172.21.15.3:0/1967387761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9b0100fd0 msgr2=0x7fa9b019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.155+0000 7fa9af400640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9b0100fd0 0x7fa9b019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.155+0000 7fa9af400640 1 -- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b01006b0 msgr2=0x7fa9b019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.155+0000 7fa9af400640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b01006b0 0x7fa9b019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.155+0000 7fa9af400640 1 -- 172.21.15.3:0/1967387761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b01a3520 con 0x7fa9b0105ac0 2024-08-01T04:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.155+0000 7fa9af400640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b01a2e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800eb10 tx=0x7fa9a800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.156+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8004b20 con 0x7fa9b0105ac0 2024-08-01T04:14:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.156+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9a8004cc0 con 0x7fa9b0105ac0 2024-08-01T04:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.156+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01a3810 con 0x7fa9b0105ac0 2024-08-01T04:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.156+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8010760 con 0x7fa9b0105ac0 2024-08-01T04:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.156+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01ab0f0 con 0x7fa9b0105ac0 2024-08-01T04:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.157+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b01054a0 con 0x7fa9b0105ac0 2024-08-01T04:14:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.159+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa9a8005020 con 0x7fa9b0105ac0 2024-08-01T04:14:17.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.160+0000 7fa9a7400640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9940a2c90 0x7fa9940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:17.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.160+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa9a80cf5b0 con 0x7fa9b0105ac0 2024-08-01T04:14:17.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.160+0000 7fa9aea00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9940a2c90 0x7fa9940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:17.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.161+0000 7fa9aea00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9940a2c90 0x7fa9940a5150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000bbe0 tx=0x7fa9a000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:17.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.164+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa9a809c460 con 0x7fa9b0105ac0 2024-08-01T04:14:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.387+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa9b0104b60 con 0x7fa9940a2c90 2024-08-01T04:14:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.425+0000 7fa9a7400640 1 -- 172.21.15.3:0/1967387761 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9b0104b60 con 0x7fa9940a2c90 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9940a2c90 msgr2=0x7fa9940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9940a2c90 0x7fa9940a5150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000bbe0 tx=0x7fa9a000b390 comp rx=0 tx=0).stop 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 msgr2=0x7fa9b01a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b01a2e40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800eb10 tx=0x7fa9a800c2d0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 shutdown_connections 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa9940a2c90 0x7fa9940a5150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9b0105ac0 0x7fa9b01a2e40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9b0100fd0 0x7fa9b019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 --2- 172.21.15.3:0/1967387761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b01006b0 0x7fa9b019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 >> 172.21.15.3:0/1967387761 conn(0x7fa9b00fc340 msgr2=0x7fa9b0103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.428+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 shutdown_connections 2024-08-01T04:14:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:17.429+0000 7fa9afe00640 1 -- 172.21.15.3:0/1967387761 wait complete. 2024-08-01T04:14:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:17 smithi003 ceph-mon[32262]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-01T04:14:17.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:17 smithi003 ceph-mon[32262]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-08-01T04:14:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:17 smithi160 ceph-mon[37224]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-01T04:14:17.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:17 smithi160 ceph-mon[37224]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-08-01T04:14:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:17 smithi163 ceph-mon[37383]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-01T04:14:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:17 smithi163 ceph-mon[37383]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-08-01T04:14:18.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-01T04:14:18.918 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:19.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:18 smithi163 ceph-mon[37383]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:19.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:18 smithi003 ceph-mon[32262]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:18 smithi160 ceph-mon[37224]: from='client.24610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 -- 172.21.15.3:0/3449457937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 msgr2=0x7f3200109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 --2- 172.21.15.3:0/3449457937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 0x7f3200109cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009860 tx=0x7f31f002a340 comp rx=0 tx=0).stop 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 -- 172.21.15.3:0/3449457937 shutdown_connections 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 --2- 172.21.15.3:0/3449457937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200104280 0x7f32001107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 --2- 172.21.15.3:0/3449457937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 --2- 172.21.15.3:0/3449457937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 0x7f3200109cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.624+0000 7f3204e00640 1 -- 172.21.15.3:0/3449457937 >> 172.21.15.3:0/3449457937 conn(0x7f32000ff5a0 msgr2=0x7f32001019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.625+0000 7f3204e00640 1 -- 172.21.15.3:0/3449457937 shutdown_connections 2024-08-01T04:14:19.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.625+0000 7f3204e00640 1 -- 172.21.15.3:0/3449457937 wait complete. 2024-08-01T04:14:19.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.626+0000 7f3204e00640 1 Processor -- start 2024-08-01T04:14:19.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.626+0000 7f3204e00640 1 -- start start 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f3204e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f3204e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200104280 0x7f3200076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f3204e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 0x7f3200076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f3204e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3200113ca0 con 0x7f32001038e0 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f3204e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3200113b20 con 0x7f32001098e0 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f3204e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3200113e20 con 0x7f3200104280 2024-08-01T04:14:19.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f31ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f31ff400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200076430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49852/0 (socket says 172.21.15.3:49852) 2024-08-01T04:14:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f31f7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200104280 0x7f3200076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f31ff400640 1 -- 172.21.15.3:0/1183306008 learned_addr learned my addr 172.21.15.3:0/1183306008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:19.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.627+0000 7f31ffe00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 0x7f3200076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31ff400640 1 -- 172.21.15.3:0/1183306008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200104280 msgr2=0x7f3200076970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31ff400640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200104280 0x7f3200076970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31ff400640 1 -- 172.21.15.3:0/1183306008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 msgr2=0x7f3200076eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31ff400640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 0x7f3200076eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31ff400640 1 -- 172.21.15.3:0/1183306008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001aa840 con 0x7f32001038e0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31ff400640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200076430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31f0033b60 tx=0x7f31f0033b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0037040 con 0x7f32001038e0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f0040070 con 0x7f32001038e0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32001aaad0 con 0x7f32001038e0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.628+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0003a10 con 0x7f32001038e0 2024-08-01T04:14:19.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.629+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32001aafb0 con 0x7f32001038e0 2024-08-01T04:14:19.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.630+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3200105020 con 0x7f32001038e0 2024-08-01T04:14:19.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.632+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f31f00041c0 con 0x7f32001038e0 2024-08-01T04:14:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.634+0000 7f31fcc00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f31e00a2c10 0x7f31e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.634+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f31f00eeab0 con 0x7f32001038e0 2024-08-01T04:14:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.634+0000 7f31f7e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f31e00a2c10 0x7f31e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:19.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.635+0000 7f31f7e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f31e00a2c10 0x7f31e00a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3200077bf0 tx=0x7f31e8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.639+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f31f00bbb70 con 0x7f32001038e0 2024-08-01T04:14:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:19 smithi003 ceph-mon[32262]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-08-01T04:14:19.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.863+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3200062360 con 0x7f31e00a2c10 2024-08-01T04:14:19.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.874+0000 7f31fcc00640 1 -- 172.21.15.3:0/1183306008 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3200062360 con 0x7f31e00a2c10 2024-08-01T04:14:19.875 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_0/f5875d78-04cf-4a36-b3b2-138cdeac6cd6 2024-08-01T04:14:19.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f31e00a2c10 msgr2=0x7f31e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f31e00a2c10 0x7f31e00a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3200077bf0 tx=0x7f31e8009260 comp rx=0 tx=0).stop 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 msgr2=0x7f3200076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200076430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31f0033b60 tx=0x7f31f0033b90 comp rx=0 tx=0).stop 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 shutdown_connections 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f31e00a2c10 0x7f31e00a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32001098e0 0x7f3200076eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3200104280 0x7f3200076970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 --2- 172.21.15.3:0/1183306008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32001038e0 0x7f3200076430 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.876+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 >> 172.21.15.3:0/1183306008 conn(0x7f32000ff5a0 msgr2=0x7f32000ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.877+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 shutdown_connections 2024-08-01T04:14:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:19.877+0000 7f3204e00640 1 -- 172.21.15.3:0/1183306008 wait complete. 2024-08-01T04:14:20.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:19 smithi163 ceph-mon[37383]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-08-01T04:14:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:19 smithi160 ceph-mon[37224]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-08-01T04:14:20.811 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-01T04:14:21.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:20 smithi163 ceph-mon[37383]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:21.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:20 smithi003 ceph-mon[32262]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:21.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:20 smithi160 ceph-mon[37224]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:21.308 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:21.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:21 smithi003 ceph-mon[32262]: pgmap v302: 97 pgs: 97 active+clean; 589 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-08-01T04:14:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 -- 172.21.15.3:0/3745972122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45901086b0 msgr2=0x7f4590114bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 --2- 172.21.15.3:0/3745972122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45901086b0 0x7f4590114bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4588009cd0 tx=0x7f458802a8e0 comp rx=0 tx=0).stop 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 -- 172.21.15.3:0/3745972122 shutdown_connections 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 --2- 172.21.15.3:0/3745972122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45901086b0 0x7f4590114bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 --2- 172.21.15.3:0/3745972122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4590107d50 0x7f4590108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 --2- 172.21.15.3:0/3745972122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f459010dd90 0x7f459010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 -- 172.21.15.3:0/3745972122 >> 172.21.15.3:0/3745972122 conn(0x7f45900776b0 msgr2=0x7f4590079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 -- 172.21.15.3:0/3745972122 shutdown_connections 2024-08-01T04:14:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.954+0000 7f4596c00640 1 -- 172.21.15.3:0/3745972122 wait complete. 2024-08-01T04:14:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.955+0000 7f4596c00640 1 Processor -- start 2024-08-01T04:14:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.955+0000 7f4596c00640 1 -- start start 2024-08-01T04:14:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4590107d50 0x7f459019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 0x7f459019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459010dd90 0x7f45901a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4590118100 con 0x7f459010dd90 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4590117f80 con 0x7f45901086b0 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4590118280 con 0x7f4590107d50 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4594e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 0x7f459019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4595800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4590107d50 0x7f459019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4596200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459010dd90 0x7f45901a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4594e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 0x7f459019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60630/0 (socket says 172.21.15.3:60630) 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4595800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4590107d50 0x7f459019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43906/0 (socket says 172.21.15.3:43906) 2024-08-01T04:14:21.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.956+0000 7f4594e00640 1 -- 172.21.15.3:0/1567388990 learned_addr learned my addr 172.21.15.3:0/1567388990 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f4594e00640 1 -- 172.21.15.3:0/1567388990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4590107d50 msgr2=0x7f459019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f4594e00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4590107d50 0x7f459019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f4594e00640 1 -- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459010dd90 msgr2=0x7f45901a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f4594e00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459010dd90 0x7f45901a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f4594e00640 1 -- 172.21.15.3:0/1567388990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45901a3650 con 0x7f45901086b0 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f4594e00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 0x7f459019ebe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9f0 tx=0x7f457c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.957+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00cd80 con 0x7f45901086b0 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.958+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45901a3940 con 0x7f45901086b0 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.958+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f457c010910 con 0x7f45901086b0 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.958+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c018990 con 0x7f45901086b0 2024-08-01T04:14:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.958+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45901ab220 con 0x7f45901086b0 2024-08-01T04:14:21.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.959+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4590108170 con 0x7f45901086b0 2024-08-01T04:14:21.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.961+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f457c010ac0 con 0x7f45901086b0 2024-08-01T04:14:21.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.963+0000 7f458e000640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f45780a2c10 0x7f45780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:21.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.963+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f457c0cf430 con 0x7f45901086b0 2024-08-01T04:14:21.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.963+0000 7f4595800640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f45780a2c10 0x7f45780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:21.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.964+0000 7f4595800640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f45780a2c10 0x7f45780a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4584009b50 tx=0x7f4584005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:21.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:21.966+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f457c09bc90 con 0x7f45901086b0 2024-08-01T04:14:22.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:21 smithi163 ceph-mon[37383]: pgmap v302: 97 pgs: 97 active+clean; 589 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-08-01T04:14:22.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:21 smithi160 ceph-mon[37224]: pgmap v302: 97 pgs: 97 active+clean; 589 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-08-01T04:14:22.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.189+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4590110250 con 0x7f45780a2c10 2024-08-01T04:14:22.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.219+0000 7f458e000640 1 -- 172.21.15.3:0/1567388990 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4590110250 con 0x7f45780a2c10 2024-08-01T04:14:22.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f45780a2c10 msgr2=0x7f45780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:22.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f45780a2c10 0x7f45780a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4584009b50 tx=0x7f4584005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:22.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 msgr2=0x7f459019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:22.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 0x7f459019ebe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f457c00e9f0 tx=0x7f457c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 shutdown_connections 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f45780a2c10 0x7f45780a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f459010dd90 0x7f45901a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45901086b0 0x7f459019ebe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.222+0000 7f4596c00640 1 --2- 172.21.15.3:0/1567388990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4590107d50 0x7f459019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.223+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 >> 172.21.15.3:0/1567388990 conn(0x7f45900776b0 msgr2=0x7f4590078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:22.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.223+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 shutdown_connections 2024-08-01T04:14:22.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:22.223+0000 7f4596c00640 1 -- 172.21.15.3:0/1567388990 wait complete. 2024-08-01T04:14:23.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-01T04:14:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:24.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:23 smithi003 ceph-mon[32262]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:24.001 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:23 smithi003 ceph-mon[32262]: pgmap v303: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-08-01T04:14:24.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:23 smithi163 ceph-mon[37383]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:24.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:23 smithi163 ceph-mon[37383]: pgmap v303: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-08-01T04:14:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:23 smithi160 ceph-mon[37224]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:23 smithi160 ceph-mon[37224]: pgmap v303: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-08-01T04:14:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 -- 172.21.15.3:0/735372112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68201038e0 msgr2=0x7f6820103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 --2- 172.21.15.3:0/735372112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68201038e0 0x7f6820103d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f680c009930 tx=0x7f680c02a420 comp rx=0 tx=0).stop 2024-08-01T04:14:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 -- 172.21.15.3:0/735372112 shutdown_connections 2024-08-01T04:14:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 --2- 172.21.15.3:0/735372112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6820104280 0x7f68201107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 --2- 172.21.15.3:0/735372112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68201038e0 0x7f6820103d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 --2- 172.21.15.3:0/735372112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68201098e0 0x7f6820109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 -- 172.21.15.3:0/735372112 >> 172.21.15.3:0/735372112 conn(0x7f68200ff5a0 msgr2=0x7f68201019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:24.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 -- 172.21.15.3:0/735372112 shutdown_connections 2024-08-01T04:14:24.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.330+0000 7f6825e00640 1 -- 172.21.15.3:0/735372112 wait complete. 2024-08-01T04:14:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 Processor -- start 2024-08-01T04:14:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 -- start start 2024-08-01T04:14:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68201038e0 0x7f682019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:24.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6820104280 0x7f682019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 0x7f682019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6820113d30 con 0x7f68201098e0 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6820113bb0 con 0x7f68201038e0 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.332+0000 7f6825e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6820113eb0 con 0x7f6820104280 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 0x7f682019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 0x7f682019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58076/0 (socket says 172.21.15.3:58076) 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 -- 172.21.15.3:0/1713977292 learned_addr learned my addr 172.21.15.3:0/1713977292 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f681fe00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6820104280 0x7f682019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6824a00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68201038e0 0x7f682019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:24.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 -- 172.21.15.3:0/1713977292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6820104280 msgr2=0x7f682019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6820104280 0x7f682019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 -- 172.21.15.3:0/1713977292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68201038e0 msgr2=0x7f682019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68201038e0 0x7f682019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.333+0000 7f6825400640 1 -- 172.21.15.3:0/1713977292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682019f1e0 con 0x7f68201098e0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.334+0000 7f6825400640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 0x7f682019eb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f681800efd0 tx=0x7f681800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.334+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818003b70 con 0x7f68201098e0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.334+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682019f4d0 con 0x7f68201098e0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.334+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818003d10 con 0x7f68201098e0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.334+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180109f0 con 0x7f68201098e0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.334+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201a6db0 con 0x7f68201098e0 2024-08-01T04:14:24.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.336+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6820105020 con 0x7f68201098e0 2024-08-01T04:14:24.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.337+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6818004020 con 0x7f68201098e0 2024-08-01T04:14:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.339+0000 7f681d600640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f68080a2c10 0x7f68080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.339+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f68180cf560 con 0x7f68201098e0 2024-08-01T04:14:24.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.339+0000 7f6824a00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f68080a2c10 0x7f68080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:24.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.340+0000 7f6824a00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f68080a2c10 0x7f68080a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6814009b50 tx=0x7f6814005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:24.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.343+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f681809c410 con 0x7f68201098e0 2024-08-01T04:14:24.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.570+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f6820062360 con 0x7f68080a2c10 2024-08-01T04:14:24.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.579+0000 7f681d600640 1 -- 172.21.15.3:0/1713977292 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6820062360 con 0x7f68080a2c10 2024-08-01T04:14:24.580 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_1/0d0a8131-aa87-46a5-b639-6265a67bbe44 2024-08-01T04:14:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f68080a2c10 msgr2=0x7f68080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f68080a2c10 0x7f68080a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6814009b50 tx=0x7f6814005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 msgr2=0x7f682019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 0x7f682019eb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f681800efd0 tx=0x7f681800c6a0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 shutdown_connections 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f68080a2c10 0x7f68080a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68201098e0 0x7f682019eb00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6820104280 0x7f682019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 --2- 172.21.15.3:0/1713977292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f68201038e0 0x7f682019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 >> 172.21.15.3:0/1713977292 conn(0x7f68200ff5a0 msgr2=0x7f68200ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 shutdown_connections 2024-08-01T04:14:24.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:24.582+0000 7f6825e00640 1 -- 172.21.15.3:0/1713977292 wait complete. 2024-08-01T04:14:25.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:14:25.894 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:26.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:25 smithi003 ceph-mon[32262]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-08-01T04:14:26.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:25 smithi003 ceph-mon[32262]: from='client.14517 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:26.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:25 smithi163 ceph-mon[37383]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-08-01T04:14:26.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:25 smithi163 ceph-mon[37383]: from='client.14517 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:26.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:25 smithi160 ceph-mon[37224]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-08-01T04:14:26.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:25 smithi160 ceph-mon[37224]: from='client.14517 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:14:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.538+0000 7f1cbda00640 1 -- 172.21.15.3:0/95363592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb81078b0 msgr2=0x7f1cb8107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.538+0000 7f1cbda00640 1 --2- 172.21.15.3:0/95363592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb81078b0 0x7f1cb8107c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009cd0 tx=0x7f1ca802a8e0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 -- 172.21.15.3:0/95363592 shutdown_connections 2024-08-01T04:14:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 --2- 172.21.15.3:0/95363592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb8102250 0x7f1cb810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 --2- 172.21.15.3:0/95363592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cb81018b0 0x7f1cb8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 --2- 172.21.15.3:0/95363592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb81078b0 0x7f1cb8107c90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 -- 172.21.15.3:0/95363592 >> 172.21.15.3:0/95363592 conn(0x7f1cb80fd5f0 msgr2=0x7f1cb80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:26.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 -- 172.21.15.3:0/95363592 shutdown_connections 2024-08-01T04:14:26.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.539+0000 7f1cbda00640 1 -- 172.21.15.3:0/95363592 wait complete. 2024-08-01T04:14:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.540+0000 7f1cbda00640 1 Processor -- start 2024-08-01T04:14:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.540+0000 7f1cbda00640 1 -- start start 2024-08-01T04:14:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.540+0000 7f1cbda00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb81018b0 0x7f1cb8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.540+0000 7f1cbda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 0x7f1cb81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.540+0000 7f1cbda00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cb81078b0 0x7f1cb819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cbda00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8111d00 con 0x7f1cb8102250 2024-08-01T04:14:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cbda00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8111b80 con 0x7f1cb81078b0 2024-08-01T04:14:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cbda00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8111e80 con 0x7f1cb81018b0 2024-08-01T04:14:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 0x7f1cb81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 0x7f1cb81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58096/0 (socket says 172.21.15.3:58096) 2024-08-01T04:14:26.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 -- 172.21.15.3:0/4073319225 learned_addr learned my addr 172.21.15.3:0/4073319225 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6a00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb81018b0 0x7f1cb8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb7400640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cb81078b0 0x7f1cb819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 -- 172.21.15.3:0/4073319225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb81018b0 msgr2=0x7f1cb8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb81018b0 0x7f1cb8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 -- 172.21.15.3:0/4073319225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cb81078b0 msgr2=0x7f1cb819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cb81078b0 0x7f1cb819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.541+0000 7f1cb6000640 1 -- 172.21.15.3:0/4073319225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb819d110 con 0x7f1cb8102250 2024-08-01T04:14:26.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.542+0000 7f1cb6000640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 0x7f1cb81986a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000e9f0 tx=0x7f1ca000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:26.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.542+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000cd80 con 0x7f1cb8102250 2024-08-01T04:14:26.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.542+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca0010910 con 0x7f1cb8102250 2024-08-01T04:14:26.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.542+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0018990 con 0x7f1cb8102250 2024-08-01T04:14:26.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.542+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb819d400 con 0x7f1cb8102250 2024-08-01T04:14:26.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.542+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81a4ce0 con 0x7f1cb8102250 2024-08-01T04:14:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.544+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb8101d10 con 0x7f1cb8102250 2024-08-01T04:14:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.546+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1ca0010ac0 con 0x7f1cb8102250 2024-08-01T04:14:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.548+0000 7f1caf400640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1c9c0a2c90 0x7f1c9c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.548+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1ca00cf390 con 0x7f1cb8102250 2024-08-01T04:14:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.548+0000 7f1cb6a00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1c9c0a2c90 0x7f1c9c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:26.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.549+0000 7f1cb6a00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1c9c0a2c90 0x7f1c9c0a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ca0 tx=0x7f1ca8009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:26.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.551+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1ca009bbf0 con 0x7f1cb8102250 2024-08-01T04:14:26.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.785+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cb8109890 con 0x7f1cb8102250 2024-08-01T04:14:26.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.788+0000 7f1caf400640 1 -- 172.21.15.3:0/4073319225 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f1ca009fef0 con 0x7f1cb8102250 2024-08-01T04:14:26.790 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:26.790 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-08-01T04:14:12:627814+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:14:11.859614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:14:26.790 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:14:26.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1c9c0a2c90 msgr2=0x7f1c9c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:26.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1c9c0a2c90 0x7f1c9c0a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009ca0 tx=0x7f1ca8009360 comp rx=0 tx=0).stop 2024-08-01T04:14:26.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 msgr2=0x7f1cb81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:26.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 0x7f1cb81986a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000e9f0 tx=0x7f1ca000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 shutdown_connections 2024-08-01T04:14:26.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1c9c0a2c90 0x7f1c9c0a5150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cb81078b0 0x7f1cb819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cb8102250 0x7f1cb81986a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 --2- 172.21.15.3:0/4073319225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb81018b0 0x7f1cb8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:26.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 >> 172.21.15.3:0/4073319225 conn(0x7f1cb80fd5f0 msgr2=0x7f1cb80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:26.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 shutdown_connections 2024-08-01T04:14:26.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:26.791+0000 7f1cbda00640 1 -- 172.21.15.3:0/4073319225 wait complete. 2024-08-01T04:14:27.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:14:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:27 smithi003 ceph-mon[32262]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-08-01T04:14:27.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:27 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/4073319225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:14:28.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:27 smithi163 ceph-mon[37383]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-08-01T04:14:28.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:27 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/4073319225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:14:28.047 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:28.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:27 smithi160 ceph-mon[37224]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-08-01T04:14:28.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:27 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/4073319225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:14:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.697+0000 7f4677200640 1 -- 172.21.15.3:0/2039818417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46781016f0 msgr2=0x7f4678101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.697+0000 7f4677200640 1 --2- 172.21.15.3:0/2039818417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46781016f0 0x7f4678101b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4664009900 tx=0x7f466402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.697+0000 7f4677200640 1 -- 172.21.15.3:0/2039818417 shutdown_connections 2024-08-01T04:14:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.697+0000 7f4677200640 1 --2- 172.21.15.3:0/2039818417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4678102090 0x7f467810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.697+0000 7f4677200640 1 --2- 172.21.15.3:0/2039818417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46781016f0 0x7f4678101b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.698+0000 7f4677200640 1 --2- 172.21.15.3:0/2039818417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46781076f0 0x7f4678107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.698+0000 7f4677200640 1 -- 172.21.15.3:0/2039818417 >> 172.21.15.3:0/2039818417 conn(0x7f46780fd470 msgr2=0x7f46780ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.698+0000 7f4677200640 1 -- 172.21.15.3:0/2039818417 shutdown_connections 2024-08-01T04:14:28.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.698+0000 7f4677200640 1 -- 172.21.15.3:0/2039818417 wait complete. 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 Processor -- start 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 -- start start 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46781016f0 0x7f4678074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 0x7f4678074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46781076f0 0x7f4678074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111b00 con 0x7f46781016f0 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111980 con 0x7f4678102090 2024-08-01T04:14:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.699+0000 7f4677200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111c80 con 0x7f46781076f0 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 0x7f4678074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46781016f0 0x7f4678074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 0x7f4678074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60704/0 (socket says 172.21.15.3:60704) 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 -- 172.21.15.3:0/3475000341 learned_addr learned my addr 172.21.15.3:0/3475000341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4676800640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46781076f0 0x7f4678074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 -- 172.21.15.3:0/3475000341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46781076f0 msgr2=0x7f4678074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46781076f0 0x7f4678074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 -- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46781016f0 msgr2=0x7f4678074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46781016f0 0x7f4678074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675400640 1 -- 172.21.15.3:0/3475000341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781a64a0 con 0x7f4678102090 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.700+0000 7f4675e00640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46781016f0 0x7f4678074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.701+0000 7f4675400640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 0x7f4678074760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f466402a9e0 tx=0x7f4664003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.701+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466403f070 con 0x7f4678102090 2024-08-01T04:14:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.701+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4664031c30 con 0x7f4678102090 2024-08-01T04:14:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.701+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664035030 con 0x7f4678102090 2024-08-01T04:14:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.701+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781a6730 con 0x7f4678102090 2024-08-01T04:14:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.701+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781a6bc0 con 0x7f4678102090 2024-08-01T04:14:28.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.702+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4678101b50 con 0x7f4678102090 2024-08-01T04:14:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.705+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4664042070 con 0x7f4678102090 2024-08-01T04:14:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.706+0000 7f466ea00640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f465c0a2c90 0x7f465c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.707+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f46640f0120 con 0x7f4678102090 2024-08-01T04:14:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.707+0000 7f4675e00640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f465c0a2c90 0x7f465c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.708+0000 7f4675e00640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f465c0a2c90 0x7f465c0a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4668005d00 tx=0x7f466800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:28.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.708+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4664049030 con 0x7f4678102090 2024-08-01T04:14:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.908+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4678109c80 con 0x7f4678102090 2024-08-01T04:14:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.910+0000 7f466ea00640 1 -- 172.21.15.3:0/3475000341 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22186 (secure 0 0 0) 0x7f4664031de0 con 0x7f4678102090 2024-08-01T04:14:28.911 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:28.912 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":86,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:13:55.770103+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f465c0a2c90 msgr2=0x7f465c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f465c0a2c90 0x7f465c0a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4668005d00 tx=0x7f466800b410 comp rx=0 tx=0).stop 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 msgr2=0x7f4678074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 0x7f4678074760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f466402a9e0 tx=0x7f4664003c10 comp rx=0 tx=0).stop 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 shutdown_connections 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f465c0a2c90 0x7f465c0a5150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46781076f0 0x7f4678074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4678102090 0x7f4678074760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 --2- 172.21.15.3:0/3475000341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46781016f0 0x7f4678074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 >> 172.21.15.3:0/3475000341 conn(0x7f46780fd470 msgr2=0x7f46780fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 shutdown_connections 2024-08-01T04:14:28.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:28.913+0000 7f4677200640 1 -- 172.21.15.3:0/3475000341 wait complete. 2024-08-01T04:14:29.772 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-01T04:14:29.772 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:14:30.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:29 smithi163 ceph-mon[37383]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-08-01T04:14:30.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:29 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3475000341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:30.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:29 smithi003 ceph-mon[32262]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-08-01T04:14:30.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:29 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3475000341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:30.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:29 smithi160 ceph-mon[37224]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-08-01T04:14:30.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:29 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3475000341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:30.217 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.857+0000 7fa3f3400640 1 -- 172.21.15.3:0/3015061325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec102a60 msgr2=0x7fa3ec102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.857+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3015061325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec102a60 0x7fa3ec102e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009930 tx=0x7fa3e002a420 comp rx=0 tx=0).stop 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.857+0000 7fa3f3400640 1 -- 172.21.15.3:0/3015061325 shutdown_connections 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.857+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3015061325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3ec107fd0 0x7fa3ec10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.857+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3015061325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec103410 0x7fa3ec1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.857+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3015061325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec102a60 0x7fa3ec102e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.858+0000 7fa3f3400640 1 -- 172.21.15.3:0/3015061325 >> 172.21.15.3:0/3015061325 conn(0x7fa3ec0fc500 msgr2=0x7fa3ec0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.858+0000 7fa3f3400640 1 -- 172.21.15.3:0/3015061325 shutdown_connections 2024-08-01T04:14:30.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.858+0000 7fa3f3400640 1 -- 172.21.15.3:0/3015061325 wait complete. 2024-08-01T04:14:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.859+0000 7fa3f3400640 1 Processor -- start 2024-08-01T04:14:30.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.859+0000 7fa3f3400640 1 -- start start 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec102a60 0x7fa3ec1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 0x7fa3ec198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3ec107fd0 0x7fa3ec19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f3400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec111e20 con 0x7fa3ec103410 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f3400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec111ca0 con 0x7fa3ec107fd0 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f3400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec111fa0 con 0x7fa3ec102a60 2024-08-01T04:14:30.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3ebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 0x7fa3ec198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:30.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec102a60 0x7fa3ec1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:30.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3ebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 0x7fa3ec198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58144/0 (socket says 172.21.15.3:58144) 2024-08-01T04:14:30.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3ebe00640 1 -- 172.21.15.3:0/3490029302 learned_addr learned my addr 172.21.15.3:0/3490029302 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:30.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.860+0000 7fa3f1400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3ec107fd0 0x7fa3ec19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3ebe00640 1 -- 172.21.15.3:0/3490029302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec102a60 msgr2=0x7fa3ec1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3ebe00640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec102a60 0x7fa3ec1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3ebe00640 1 -- 172.21.15.3:0/3490029302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3ec107fd0 msgr2=0x7fa3ec19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3ebe00640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3ec107fd0 0x7fa3ec19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3ebe00640 1 -- 172.21.15.3:0/3490029302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec19d270 con 0x7fa3ec103410 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3f0a00640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec102a60 0x7fa3ec1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3ebe00640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 0x7fa3ec198800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800e9f0 tx=0x7fa3d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800ccc0 con 0x7fa3ec103410 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8010910 con 0x7fa3ec103410 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec19d560 con 0x7fa3ec103410 2024-08-01T04:14:30.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.861+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8018920 con 0x7fa3ec103410 2024-08-01T04:14:30.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.862+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec1a4e40 con 0x7fa3ec103410 2024-08-01T04:14:30.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.863+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ec06a580 con 0x7fa3ec103410 2024-08-01T04:14:30.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.865+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa3d8010ac0 con 0x7fa3ec103410 2024-08-01T04:14:30.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.867+0000 7fa3e9600640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3d40a2c10 0x7fa3d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.867+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa3d80cf3b0 con 0x7fa3ec103410 2024-08-01T04:14:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.867+0000 7fa3f0a00640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3d40a2c10 0x7fa3d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:30.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.868+0000 7fa3f0a00640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3d40a2c10 0x7fa3d40a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0006010 tx=0x7fa3e0031510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:30.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:30.871+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa3d809bc10 con 0x7fa3ec103410 2024-08-01T04:14:31.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.064+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3ec062360 con 0x7fa3d40a2c10 2024-08-01T04:14:31.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.087+0000 7fa3e9600640 1 -- 172.21.15.3:0/3490029302 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258825 (secure 0 0 0) 0x7fa3ec062360 con 0x7fa3d40a2c10 2024-08-01T04:14:31.097 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:31.098 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:14:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3d40a2c10 msgr2=0x7fa3d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3d40a2c10 0x7fa3d40a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0006010 tx=0x7fa3e0031510 comp rx=0 tx=0).stop 2024-08-01T04:14:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 msgr2=0x7fa3ec198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 0x7fa3ec198800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800e9f0 tx=0x7fa3d800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:14:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 shutdown_connections 2024-08-01T04:14:31.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3d40a2c10 0x7fa3d40a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3ec107fd0 0x7fa3ec19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.093+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3ec103410 0x7fa3ec198800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.094+0000 7fa3f3400640 1 --2- 172.21.15.3:0/3490029302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3ec102a60 0x7fa3ec1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.094+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 >> 172.21.15.3:0/3490029302 conn(0x7fa3ec0fc500 msgr2=0x7fa3ec105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.094+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 shutdown_connections 2024-08-01T04:14:31.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:31.094+0000 7fa3f3400640 1 -- 172.21.15.3:0/3490029302 wait complete. 2024-08-01T04:14:31.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:31 smithi003 ceph-mon[32262]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-08-01T04:14:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:31 smithi163 ceph-mon[37383]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-08-01T04:14:32.056 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-08-01T04:14:30.197931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651931,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":273,"num_read_kb":250,"num_write":342,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364712,"kb_used_data":11400,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577112,"statfs":{"total":1151940427776,"available":1151566962688,"internally_reserved":0,"allocated":11673600,"data_stored":7337622,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49352,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":80,"num_write":27,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528883+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:13:56.528883+0000","last_peered":"2024-08-01T04:13:56.528883+0000","last_clean":"2024-08-01T04:13:56.528883+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:13:56.528883+0000","last_undegraded":"2024-08-01T04:13:56.528883+0000","last_fullsized":"2024-08-01T04:13:56.528883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.537905+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:13:56.537905+0000","last_peered":"2024-08-01T04:13:56.537905+0000","last_clean":"2024-08-01T04:13:56.537905+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:13:56.537905+0000","last_undegraded":"2024-08-01T04:13:56.537905+0000","last_fullsized":"2024-08-01T04:13:56.537905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.537852+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:13:56.537852+0000","last_peered":"2024-08-01T04:13:56.537852+0000","last_clean":"2024-08-01T04:13:56.537852+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:13:56.537852+0000","last_undegraded":"2024-08-01T04:13:56.537852+0000","last_fullsized":"2024-08-01T04:13:56.537852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.505984+0000","last_change":"2024-08-01T04:13:54.507379+0000","last_active":"2024-08-01T04:13:56.505984+0000","last_peered":"2024-08-01T04:13:56.505984+0000","last_clean":"2024-08-01T04:13:56.505984+0000","last_became_active":"2024-08-01T04:13:54.507066+0000","last_became_peered":"2024-08-01T04:13:54.507066+0000","last_unstale":"2024-08-01T04:13:56.505984+0000","last_undegraded":"2024-08-01T04:13:56.505984+0000","last_fullsized":"2024-08-01T04:13:56.505984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:54.117496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529184+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:13:56.529184+0000","last_peered":"2024-08-01T04:13:56.529184+0000","last_clean":"2024-08-01T04:13:56.529184+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:13:56.529184+0000","last_undegraded":"2024-08-01T04:13:56.529184+0000","last_fullsized":"2024-08-01T04:13:56.529184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528181+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:13:56.528181+0000","last_peered":"2024-08-01T04:13:56.528181+0000","last_clean":"2024-08-01T04:13:56.528181+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:13:56.528181+0000","last_undegraded":"2024-08-01T04:13:56.528181+0000","last_fullsized":"2024-08-01T04:13:56.528181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526976+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:13:56.526976+0000","last_peered":"2024-08-01T04:13:56.526976+0000","last_clean":"2024-08-01T04:13:56.526976+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:13:56.526976+0000","last_undegraded":"2024-08-01T04:13:56.526976+0000","last_fullsized":"2024-08-01T04:13:56.526976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.508951+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:13:56.508951+0000","last_peered":"2024-08-01T04:13:56.508951+0000","last_clean":"2024-08-01T04:13:56.508951+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:13:56.508951+0000","last_undegraded":"2024-08-01T04:13:56.508951+0000","last_fullsized":"2024-08-01T04:13:56.508951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531349+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:13:56.531349+0000","last_peered":"2024-08-01T04:13:56.531349+0000","last_clean":"2024-08-01T04:13:56.531349+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:13:56.531349+0000","last_undegraded":"2024-08-01T04:13:56.531349+0000","last_fullsized":"2024-08-01T04:13:56.531349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529013+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:13:56.529013+0000","last_peered":"2024-08-01T04:13:56.529013+0000","last_clean":"2024-08-01T04:13:56.529013+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:13:56.529013+0000","last_undegraded":"2024-08-01T04:13:56.529013+0000","last_fullsized":"2024-08-01T04:13:56.529013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528977+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:13:56.528977+0000","last_peered":"2024-08-01T04:13:56.528977+0000","last_clean":"2024-08-01T04:13:56.528977+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:13:56.528977+0000","last_undegraded":"2024-08-01T04:13:56.528977+0000","last_fullsized":"2024-08-01T04:13:56.528977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506072+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:13:56.506072+0000","last_peered":"2024-08-01T04:13:56.506072+0000","last_clean":"2024-08-01T04:13:56.506072+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:13:56.506072+0000","last_undegraded":"2024-08-01T04:13:56.506072+0000","last_fullsized":"2024-08-01T04:13:56.506072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530326+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:13:56.530326+0000","last_peered":"2024-08-01T04:13:56.530326+0000","last_clean":"2024-08-01T04:13:56.530326+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:13:56.530326+0000","last_undegraded":"2024-08-01T04:13:56.530326+0000","last_fullsized":"2024-08-01T04:13:56.530326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.536358+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:13:56.536358+0000","last_peered":"2024-08-01T04:13:56.536358+0000","last_clean":"2024-08-01T04:13:56.536358+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:13:56.536358+0000","last_undegraded":"2024-08-01T04:13:56.536358+0000","last_fullsized":"2024-08-01T04:13:56.536358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.536382+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:13:56.536382+0000","last_peered":"2024-08-01T04:13:56.536382+0000","last_clean":"2024-08-01T04:13:56.536382+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:13:56.536382+0000","last_undegraded":"2024-08-01T04:13:56.536382+0000","last_fullsized":"2024-08-01T04:13:56.536382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530428+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:13:56.530428+0000","last_peered":"2024-08-01T04:13:56.530428+0000","last_clean":"2024-08-01T04:13:56.530428+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:13:56.530428+0000","last_undegraded":"2024-08-01T04:13:56.530428+0000","last_fullsized":"2024-08-01T04:13:56.530428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529079+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:13:56.529079+0000","last_peered":"2024-08-01T04:13:56.529079+0000","last_clean":"2024-08-01T04:13:56.529079+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:13:56.529079+0000","last_undegraded":"2024-08-01T04:13:56.529079+0000","last_fullsized":"2024-08-01T04:13:56.529079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.532924+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:13:56.532924+0000","last_peered":"2024-08-01T04:13:56.532924+0000","last_clean":"2024-08-01T04:13:56.532924+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:13:56.532924+0000","last_undegraded":"2024-08-01T04:13:56.532924+0000","last_fullsized":"2024-08-01T04:13:56.532924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527186+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:13:56.527186+0000","last_peered":"2024-08-01T04:13:56.527186+0000","last_clean":"2024-08-01T04:13:56.527186+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:13:56.527186+0000","last_undegraded":"2024-08-01T04:13:56.527186+0000","last_fullsized":"2024-08-01T04:13:56.527186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528008+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:13:56.528008+0000","last_peered":"2024-08-01T04:13:56.528008+0000","last_clean":"2024-08-01T04:13:56.528008+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:13:56.528008+0000","last_undegraded":"2024-08-01T04:13:56.528008+0000","last_fullsized":"2024-08-01T04:13:56.528008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530818+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:13:56.530818+0000","last_peered":"2024-08-01T04:13:56.530818+0000","last_clean":"2024-08-01T04:13:56.530818+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:13:56.530818+0000","last_undegraded":"2024-08-01T04:13:56.530818+0000","last_fullsized":"2024-08-01T04:13:56.530818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531085+0000","last_change":"2024-08-01T04:13:54.526711+0000","last_active":"2024-08-01T04:13:56.531085+0000","last_peered":"2024-08-01T04:13:56.531085+0000","last_clean":"2024-08-01T04:13:56.531085+0000","last_became_active":"2024-08-01T04:13:54.526371+0000","last_became_peered":"2024-08-01T04:13:54.526371+0000","last_unstale":"2024-08-01T04:13:56.531085+0000","last_undegraded":"2024-08-01T04:13:56.531085+0000","last_fullsized":"2024-08-01T04:13:56.531085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:24:06.453248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.508965+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:13:56.508965+0000","last_peered":"2024-08-01T04:13:56.508965+0000","last_clean":"2024-08-01T04:13:56.508965+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:13:56.508965+0000","last_undegraded":"2024-08-01T04:13:56.508965+0000","last_fullsized":"2024-08-01T04:13:56.508965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506870+0000","last_change":"2024-08-01T04:13:54.521065+0000","last_active":"2024-08-01T04:13:56.506870+0000","last_peered":"2024-08-01T04:13:56.506870+0000","last_clean":"2024-08-01T04:13:56.506870+0000","last_became_active":"2024-08-01T04:13:54.520527+0000","last_became_peered":"2024-08-01T04:13:54.520527+0000","last_unstale":"2024-08-01T04:13:56.506870+0000","last_undegraded":"2024-08-01T04:13:56.506870+0000","last_fullsized":"2024-08-01T04:13:56.506870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:43.132148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.510396+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:13:56.510396+0000","last_peered":"2024-08-01T04:13:56.510396+0000","last_clean":"2024-08-01T04:13:56.510396+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:13:56.510396+0000","last_undegraded":"2024-08-01T04:13:56.510396+0000","last_fullsized":"2024-08-01T04:13:56.510396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527435+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:13:56.527435+0000","last_peered":"2024-08-01T04:13:56.527435+0000","last_clean":"2024-08-01T04:13:56.527435+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:13:56.527435+0000","last_undegraded":"2024-08-01T04:13:56.527435+0000","last_fullsized":"2024-08-01T04:13:56.527435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528523+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:13:56.528523+0000","last_peered":"2024-08-01T04:13:56.528523+0000","last_clean":"2024-08-01T04:13:56.528523+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:13:56.528523+0000","last_undegraded":"2024-08-01T04:13:56.528523+0000","last_fullsized":"2024-08-01T04:13:56.528523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529695+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:13:56.529695+0000","last_peered":"2024-08-01T04:13:56.529695+0000","last_clean":"2024-08-01T04:13:56.529695+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:13:56.529695+0000","last_undegraded":"2024-08-01T04:13:56.529695+0000","last_fullsized":"2024-08-01T04:13:56.529695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.537002+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:13:56.537002+0000","last_peered":"2024-08-01T04:13:56.537002+0000","last_clean":"2024-08-01T04:13:56.537002+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:13:56.537002+0000","last_undegraded":"2024-08-01T04:13:56.537002+0000","last_fullsized":"2024-08-01T04:13:56.537002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526355+0000","last_change":"2024-08-01T04:13:54.526099+0000","last_active":"2024-08-01T04:13:56.526355+0000","last_peered":"2024-08-01T04:13:56.526355+0000","last_clean":"2024-08-01T04:13:56.526355+0000","last_became_active":"2024-08-01T04:13:54.525730+0000","last_became_peered":"2024-08-01T04:13:54.525730+0000","last_unstale":"2024-08-01T04:13:56.526355+0000","last_undegraded":"2024-08-01T04:13:56.526355+0000","last_fullsized":"2024-08-01T04:13:56.526355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:17:40.338865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527049+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:13:56.527049+0000","last_peered":"2024-08-01T04:13:56.527049+0000","last_clean":"2024-08-01T04:13:56.527049+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:13:56.527049+0000","last_undegraded":"2024-08-01T04:13:56.527049+0000","last_fullsized":"2024-08-01T04:13:56.527049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526947+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:13:56.526947+0000","last_peered":"2024-08-01T04:13:56.526947+0000","last_clean":"2024-08-01T04:13:56.526947+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:13:56.526947+0000","last_undegraded":"2024-08-01T04:13:56.526947+0000","last_fullsized":"2024-08-01T04:13:56.526947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529708+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:13:56.529708+0000","last_peered":"2024-08-01T04:13:56.529708+0000","last_clean":"2024-08-01T04:13:56.529708+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:13:56.529708+0000","last_undegraded":"2024-08-01T04:13:56.529708+0000","last_fullsized":"2024-08-01T04:13:56.529708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:27.657855+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:14:27.657855+0000","last_peered":"2024-08-01T04:14:27.657855+0000","last_clean":"2024-08-01T04:14:27.657855+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:14:27.657855+0000","last_undegraded":"2024-08-01T04:14:27.657855+0000","last_fullsized":"2024-08-01T04:14:27.657855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.507073+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:13:56.507073+0000","last_peered":"2024-08-01T04:13:56.507073+0000","last_clean":"2024-08-01T04:13:56.507073+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:13:56.507073+0000","last_undegraded":"2024-08-01T04:13:56.507073+0000","last_fullsized":"2024-08-01T04:13:56.507073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.609507+0000","last_change":"2024-08-01T04:13:51.458870+0000","last_active":"2024-08-01T04:14:07.609507+0000","last_peered":"2024-08-01T04:14:07.609507+0000","last_clean":"2024-08-01T04:14:07.609507+0000","last_became_active":"2024-08-01T04:13:51.458507+0000","last_became_peered":"2024-08-01T04:13:51.458507+0000","last_unstale":"2024-08-01T04:14:07.609507+0000","last_undegraded":"2024-08-01T04:14:07.609507+0000","last_fullsized":"2024-08-01T04:14:07.609507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:21:57.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.611983+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:14:07.611983+0000","last_peered":"2024-08-01T04:14:07.611983+0000","last_clean":"2024-08-01T04:14:07.611983+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:14:07.611983+0000","last_undegraded":"2024-08-01T04:14:07.611983+0000","last_fullsized":"2024-08-01T04:14:07.611983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.509085+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:13:56.509085+0000","last_peered":"2024-08-01T04:13:56.509085+0000","last_clean":"2024-08-01T04:13:56.509085+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:13:56.509085+0000","last_undegraded":"2024-08-01T04:13:56.509085+0000","last_fullsized":"2024-08-01T04:13:56.509085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.507426+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:13:56.507426+0000","last_peered":"2024-08-01T04:13:56.507426+0000","last_clean":"2024-08-01T04:13:56.507426+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:13:56.507426+0000","last_undegraded":"2024-08-01T04:13:56.507426+0000","last_fullsized":"2024-08-01T04:13:56.507426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.532080+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:13:56.532080+0000","last_peered":"2024-08-01T04:13:56.532080+0000","last_clean":"2024-08-01T04:13:56.532080+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:13:56.532080+0000","last_undegraded":"2024-08-01T04:13:56.532080+0000","last_fullsized":"2024-08-01T04:13:56.532080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:25.654119+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:14:25.654119+0000","last_peered":"2024-08-01T04:14:25.654119+0000","last_clean":"2024-08-01T04:14:25.654119+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:14:25.654119+0000","last_undegraded":"2024-08-01T04:14:25.654119+0000","last_fullsized":"2024-08-01T04:14:25.654119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531153+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:13:56.531153+0000","last_peered":"2024-08-01T04:13:56.531153+0000","last_clean":"2024-08-01T04:13:56.531153+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:13:56.531153+0000","last_undegraded":"2024-08-01T04:13:56.531153+0000","last_fullsized":"2024-08-01T04:13:56.531153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.602386+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:14:06.602386+0000","last_peered":"2024-08-01T04:14:06.602386+0000","last_clean":"2024-08-01T04:14:06.602386+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:14:06.602386+0000","last_undegraded":"2024-08-01T04:14:06.602386+0000","last_fullsized":"2024-08-01T04:14:06.602386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.538542+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:13:56.538542+0000","last_peered":"2024-08-01T04:13:56.538542+0000","last_clean":"2024-08-01T04:13:56.538542+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:13:56.538542+0000","last_undegraded":"2024-08-01T04:13:56.538542+0000","last_fullsized":"2024-08-01T04:13:56.538542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.601022+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:14:06.601022+0000","last_peered":"2024-08-01T04:14:06.601022+0000","last_clean":"2024-08-01T04:14:06.601022+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:14:06.601022+0000","last_undegraded":"2024-08-01T04:14:06.601022+0000","last_fullsized":"2024-08-01T04:14:06.601022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531094+0000","last_change":"2024-08-01T04:13:54.503730+0000","last_active":"2024-08-01T04:13:56.531094+0000","last_peered":"2024-08-01T04:13:56.531094+0000","last_clean":"2024-08-01T04:13:56.531094+0000","last_became_active":"2024-08-01T04:13:54.503412+0000","last_became_peered":"2024-08-01T04:13:54.503412+0000","last_unstale":"2024-08-01T04:13:56.531094+0000","last_undegraded":"2024-08-01T04:13:56.531094+0000","last_fullsized":"2024-08-01T04:13:56.531094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:12:48.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":90,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:26.656518+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:14:26.656518+0000","last_peered":"2024-08-01T04:14:26.656518+0000","last_clean":"2024-08-01T04:14:26.656518+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:14:26.656518+0000","last_undegraded":"2024-08-01T04:14:26.656518+0000","last_fullsized":"2024-08-01T04:14:26.656518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56939,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":76,"num_write":27,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529527+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:13:56.529527+0000","last_peered":"2024-08-01T04:13:56.529527+0000","last_clean":"2024-08-01T04:13:56.529527+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:13:56.529527+0000","last_undegraded":"2024-08-01T04:13:56.529527+0000","last_fullsized":"2024-08-01T04:13:56.529527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.606498+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:14:07.606498+0000","last_peered":"2024-08-01T04:14:07.606498+0000","last_clean":"2024-08-01T04:14:07.606498+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:14:07.606498+0000","last_undegraded":"2024-08-01T04:14:07.606498+0000","last_fullsized":"2024-08-01T04:14:07.606498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529575+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:13:56.529575+0000","last_peered":"2024-08-01T04:13:56.529575+0000","last_clean":"2024-08-01T04:13:56.529575+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:13:56.529575+0000","last_undegraded":"2024-08-01T04:13:56.529575+0000","last_fullsized":"2024-08-01T04:13:56.529575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605303+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:14:06.605303+0000","last_peered":"2024-08-01T04:14:06.605303+0000","last_clean":"2024-08-01T04:14:06.605303+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:14:06.605303+0000","last_undegraded":"2024-08-01T04:14:06.605303+0000","last_fullsized":"2024-08-01T04:14:06.605303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527463+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:13:56.527463+0000","last_peered":"2024-08-01T04:13:56.527463+0000","last_clean":"2024-08-01T04:13:56.527463+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:13:56.527463+0000","last_undegraded":"2024-08-01T04:13:56.527463+0000","last_fullsized":"2024-08-01T04:13:56.527463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:26.660306+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:14:26.660306+0000","last_peered":"2024-08-01T04:14:26.660306+0000","last_clean":"2024-08-01T04:14:26.660306+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:14:26.660306+0000","last_undegraded":"2024-08-01T04:14:26.660306+0000","last_fullsized":"2024-08-01T04:14:26.660306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526833+0000","last_change":"2024-08-01T04:13:54.509611+0000","last_active":"2024-08-01T04:13:56.526833+0000","last_peered":"2024-08-01T04:13:56.526833+0000","last_clean":"2024-08-01T04:13:56.526833+0000","last_became_active":"2024-08-01T04:13:54.509246+0000","last_became_peered":"2024-08-01T04:13:54.509246+0000","last_unstale":"2024-08-01T04:13:56.526833+0000","last_undegraded":"2024-08-01T04:13:56.526833+0000","last_fullsized":"2024-08-01T04:13:56.526833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:15:08.754404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.609796+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:14:07.609796+0000","last_peered":"2024-08-01T04:14:07.609796+0000","last_clean":"2024-08-01T04:14:07.609796+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:14:07.609796+0000","last_undegraded":"2024-08-01T04:14:07.609796+0000","last_fullsized":"2024-08-01T04:14:07.609796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506977+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:13:56.506977+0000","last_peered":"2024-08-01T04:13:56.506977+0000","last_clean":"2024-08-01T04:13:56.506977+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:13:56.506977+0000","last_undegraded":"2024-08-01T04:13:56.506977+0000","last_fullsized":"2024-08-01T04:13:56.506977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:09.599827+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:14:09.599827+0000","last_peered":"2024-08-01T04:14:09.599827+0000","last_clean":"2024-08-01T04:14:09.599827+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:14:09.599827+0000","last_undegraded":"2024-08-01T04:14:09.599827+0000","last_fullsized":"2024-08-01T04:14:09.599827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528052+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:13:56.528052+0000","last_peered":"2024-08-01T04:13:56.528052+0000","last_clean":"2024-08-01T04:13:56.528052+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:13:56.528052+0000","last_undegraded":"2024-08-01T04:13:56.528052+0000","last_fullsized":"2024-08-01T04:13:56.528052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.534081+0000","last_change":"2024-08-01T04:13:51.466500+0000","last_active":"2024-08-01T04:13:56.534081+0000","last_peered":"2024-08-01T04:13:56.534081+0000","last_clean":"2024-08-01T04:13:56.534081+0000","last_became_active":"2024-08-01T04:13:51.466069+0000","last_became_peered":"2024-08-01T04:13:51.466069+0000","last_unstale":"2024-08-01T04:13:56.534081+0000","last_undegraded":"2024-08-01T04:13:56.534081+0000","last_fullsized":"2024-08-01T04:13:56.534081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:30:03.961179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528594+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:13:56.528594+0000","last_peered":"2024-08-01T04:13:56.528594+0000","last_clean":"2024-08-01T04:13:56.528594+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:13:56.528594+0000","last_undegraded":"2024-08-01T04:13:56.528594+0000","last_fullsized":"2024-08-01T04:13:56.528594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530578+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:13:56.530578+0000","last_peered":"2024-08-01T04:13:56.530578+0000","last_clean":"2024-08-01T04:13:56.530578+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:13:56.530578+0000","last_undegraded":"2024-08-01T04:13:56.530578+0000","last_fullsized":"2024-08-01T04:13:56.530578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527814+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:13:56.527814+0000","last_peered":"2024-08-01T04:13:56.527814+0000","last_clean":"2024-08-01T04:13:56.527814+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:13:56.527814+0000","last_undegraded":"2024-08-01T04:13:56.527814+0000","last_fullsized":"2024-08-01T04:13:56.527814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.606867+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:14:07.606867+0000","last_peered":"2024-08-01T04:14:07.606867+0000","last_clean":"2024-08-01T04:14:07.606867+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:14:07.606867+0000","last_undegraded":"2024-08-01T04:14:07.606867+0000","last_fullsized":"2024-08-01T04:14:07.606867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529210+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:13:56.529210+0000","last_peered":"2024-08-01T04:13:56.529210+0000","last_clean":"2024-08-01T04:13:56.529210+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:13:56.529210+0000","last_undegraded":"2024-08-01T04:13:56.529210+0000","last_fullsized":"2024-08-01T04:13:56.529210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527936+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:13:56.527936+0000","last_peered":"2024-08-01T04:13:56.527936+0000","last_clean":"2024-08-01T04:13:56.527936+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:13:56.527936+0000","last_undegraded":"2024-08-01T04:13:56.527936+0000","last_fullsized":"2024-08-01T04:13:56.527936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:09.624388+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:14:09.624388+0000","last_peered":"2024-08-01T04:14:09.624388+0000","last_clean":"2024-08-01T04:14:09.624388+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:14:09.624388+0000","last_undegraded":"2024-08-01T04:14:09.624388+0000","last_fullsized":"2024-08-01T04:14:09.624388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.509865+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:13:56.509865+0000","last_peered":"2024-08-01T04:13:56.509865+0000","last_clean":"2024-08-01T04:13:56.509865+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:13:56.509865+0000","last_undegraded":"2024-08-01T04:13:56.509865+0000","last_fullsized":"2024-08-01T04:13:56.509865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.602542+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:14:07.602542+0000","last_peered":"2024-08-01T04:14:07.602542+0000","last_clean":"2024-08-01T04:14:07.602542+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:14:07.602542+0000","last_undegraded":"2024-08-01T04:14:07.602542+0000","last_fullsized":"2024-08-01T04:14:07.602542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529278+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:13:56.529278+0000","last_peered":"2024-08-01T04:13:56.529278+0000","last_clean":"2024-08-01T04:13:56.529278+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:13:56.529278+0000","last_undegraded":"2024-08-01T04:13:56.529278+0000","last_fullsized":"2024-08-01T04:13:56.529278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:24.656113+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:14:24.656113+0000","last_peered":"2024-08-01T04:14:24.656113+0000","last_clean":"2024-08-01T04:14:24.656113+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:14:24.656113+0000","last_undegraded":"2024-08-01T04:14:24.656113+0000","last_fullsized":"2024-08-01T04:14:24.656113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506964+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:13:56.506964+0000","last_peered":"2024-08-01T04:13:56.506964+0000","last_clean":"2024-08-01T04:13:56.506964+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:13:56.506964+0000","last_undegraded":"2024-08-01T04:13:56.506964+0000","last_fullsized":"2024-08-01T04:13:56.506964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527343+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:13:56.527343+0000","last_peered":"2024-08-01T04:13:56.527343+0000","last_clean":"2024-08-01T04:13:56.527343+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:13:56.527343+0000","last_undegraded":"2024-08-01T04:13:56.527343+0000","last_fullsized":"2024-08-01T04:13:56.527343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528022+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:13:56.528022+0000","last_peered":"2024-08-01T04:13:56.528022+0000","last_clean":"2024-08-01T04:13:56.528022+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:13:56.528022+0000","last_undegraded":"2024-08-01T04:13:56.528022+0000","last_fullsized":"2024-08-01T04:13:56.528022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.532665+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:13:56.532665+0000","last_peered":"2024-08-01T04:13:56.532665+0000","last_clean":"2024-08-01T04:13:56.532665+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:13:56.532665+0000","last_undegraded":"2024-08-01T04:13:56.532665+0000","last_fullsized":"2024-08-01T04:13:56.532665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.921198+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:13:56.921198+0000","last_peered":"2024-08-01T04:13:56.921198+0000","last_clean":"2024-08-01T04:13:56.921198+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:13:56.921198+0000","last_undegraded":"2024-08-01T04:13:56.921198+0000","last_fullsized":"2024-08-01T04:13:56.921198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605918+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:14:06.605918+0000","last_peered":"2024-08-01T04:14:06.605918+0000","last_clean":"2024-08-01T04:14:06.605918+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:14:06.605918+0000","last_undegraded":"2024-08-01T04:14:06.605918+0000","last_fullsized":"2024-08-01T04:14:06.605918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529861+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:13:56.529861+0000","last_peered":"2024-08-01T04:13:56.529861+0000","last_clean":"2024-08-01T04:13:56.529861+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:13:56.529861+0000","last_undegraded":"2024-08-01T04:13:56.529861+0000","last_fullsized":"2024-08-01T04:13:56.529861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:25.530465+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:14:25.530465+0000","last_peered":"2024-08-01T04:14:25.530465+0000","last_clean":"2024-08-01T04:14:25.530465+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:14:25.530465+0000","last_undegraded":"2024-08-01T04:14:25.530465+0000","last_fullsized":"2024-08-01T04:14:25.530465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.510446+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:13:56.510446+0000","last_peered":"2024-08-01T04:13:56.510446+0000","last_clean":"2024-08-01T04:13:56.510446+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:13:56.510446+0000","last_undegraded":"2024-08-01T04:13:56.510446+0000","last_fullsized":"2024-08-01T04:13:56.510446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605467+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:14:06.605467+0000","last_peered":"2024-08-01T04:14:06.605467+0000","last_clean":"2024-08-01T04:14:06.605467+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:14:06.605467+0000","last_undegraded":"2024-08-01T04:14:06.605467+0000","last_fullsized":"2024-08-01T04:14:06.605467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.538183+0000","last_change":"2024-08-01T04:13:54.532925+0000","last_active":"2024-08-01T04:13:56.538183+0000","last_peered":"2024-08-01T04:13:56.538183+0000","last_clean":"2024-08-01T04:13:56.538183+0000","last_became_active":"2024-08-01T04:13:54.531886+0000","last_became_peered":"2024-08-01T04:13:54.531886+0000","last_unstale":"2024-08-01T04:13:56.538183+0000","last_undegraded":"2024-08-01T04:13:56.538183+0000","last_fullsized":"2024-08-01T04:13:56.538183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:39.471789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.602793+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:14:06.602793+0000","last_peered":"2024-08-01T04:14:06.602793+0000","last_clean":"2024-08-01T04:14:06.602793+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:14:06.602793+0000","last_undegraded":"2024-08-01T04:14:06.602793+0000","last_fullsized":"2024-08-01T04:14:06.602793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531055+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:13:56.531055+0000","last_peered":"2024-08-01T04:13:56.531055+0000","last_clean":"2024-08-01T04:13:56.531055+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:13:56.531055+0000","last_undegraded":"2024-08-01T04:13:56.531055+0000","last_fullsized":"2024-08-01T04:13:56.531055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605844+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:14:06.605844+0000","last_peered":"2024-08-01T04:14:06.605844+0000","last_clean":"2024-08-01T04:14:06.605844+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:14:06.605844+0000","last_undegraded":"2024-08-01T04:14:06.605844+0000","last_fullsized":"2024-08-01T04:14:06.605844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527722+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:13:56.527722+0000","last_peered":"2024-08-01T04:13:56.527722+0000","last_clean":"2024-08-01T04:13:56.527722+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:13:56.527722+0000","last_undegraded":"2024-08-01T04:13:56.527722+0000","last_fullsized":"2024-08-01T04:13:56.527722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526657+0000","last_change":"2024-08-01T04:13:51.465957+0000","last_active":"2024-08-01T04:13:56.526657+0000","last_peered":"2024-08-01T04:13:56.526657+0000","last_clean":"2024-08-01T04:13:56.526657+0000","last_became_active":"2024-08-01T04:13:51.465550+0000","last_became_peered":"2024-08-01T04:13:51.465550+0000","last_unstale":"2024-08-01T04:13:56.526657+0000","last_undegraded":"2024-08-01T04:13:56.526657+0000","last_fullsized":"2024-08-01T04:13:56.526657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:32:20.552737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.509429+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:13:56.509429+0000","last_peered":"2024-08-01T04:13:56.509429+0000","last_clean":"2024-08-01T04:13:56.509429+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:13:56.509429+0000","last_undegraded":"2024-08-01T04:13:56.509429+0000","last_fullsized":"2024-08-01T04:13:56.509429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.602064+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:14:06.602064+0000","last_peered":"2024-08-01T04:14:06.602064+0000","last_clean":"2024-08-01T04:14:06.602064+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:14:06.602064+0000","last_undegraded":"2024-08-01T04:14:06.602064+0000","last_fullsized":"2024-08-01T04:14:06.602064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527081+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:13:56.527081+0000","last_peered":"2024-08-01T04:13:56.527081+0000","last_clean":"2024-08-01T04:13:56.527081+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:13:56.527081+0000","last_undegraded":"2024-08-01T04:13:56.527081+0000","last_fullsized":"2024-08-01T04:13:56.527081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.507217+0000","last_change":"2024-08-01T04:13:54.503954+0000","last_active":"2024-08-01T04:13:56.507217+0000","last_peered":"2024-08-01T04:13:56.507217+0000","last_clean":"2024-08-01T04:13:56.507217+0000","last_became_active":"2024-08-01T04:13:54.503613+0000","last_became_peered":"2024-08-01T04:13:54.503613+0000","last_unstale":"2024-08-01T04:13:56.507217+0000","last_undegraded":"2024-08-01T04:13:56.507217+0000","last_fullsized":"2024-08-01T04:13:56.507217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:19:01.729326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.534411+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:13:56.534411+0000","last_peered":"2024-08-01T04:13:56.534411+0000","last_clean":"2024-08-01T04:13:56.534411+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:13:56.534411+0000","last_undegraded":"2024-08-01T04:13:56.534411+0000","last_fullsized":"2024-08-01T04:13:56.534411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530782+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:13:56.530782+0000","last_peered":"2024-08-01T04:13:56.530782+0000","last_clean":"2024-08-01T04:13:56.530782+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:13:56.530782+0000","last_undegraded":"2024-08-01T04:13:56.530782+0000","last_fullsized":"2024-08-01T04:13:56.530782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.602238+0000","last_change":"2024-08-01T04:13:51.461259+0000","last_active":"2024-08-01T04:14:07.602238+0000","last_peered":"2024-08-01T04:14:07.602238+0000","last_clean":"2024-08-01T04:14:07.602238+0000","last_became_active":"2024-08-01T04:13:51.460884+0000","last_became_peered":"2024-08-01T04:13:51.460884+0000","last_unstale":"2024-08-01T04:14:07.602238+0000","last_undegraded":"2024-08-01T04:14:07.602238+0000","last_fullsized":"2024-08-01T04:14:07.602238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:18.454778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.538110+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:13:56.538110+0000","last_peered":"2024-08-01T04:13:56.538110+0000","last_clean":"2024-08-01T04:13:56.538110+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:13:56.538110+0000","last_undegraded":"2024-08-01T04:13:56.538110+0000","last_fullsized":"2024-08-01T04:13:56.538110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.535463+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:13:56.535463+0000","last_peered":"2024-08-01T04:13:56.535463+0000","last_clean":"2024-08-01T04:13:56.535463+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:13:56.535463+0000","last_undegraded":"2024-08-01T04:13:56.535463+0000","last_fullsized":"2024-08-01T04:13:56.535463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.510546+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:13:56.510546+0000","last_peered":"2024-08-01T04:13:56.510546+0000","last_clean":"2024-08-01T04:13:56.510546+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:13:56.510546+0000","last_undegraded":"2024-08-01T04:13:56.510546+0000","last_fullsized":"2024-08-01T04:13:56.510546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.611180+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:14:07.611180+0000","last_peered":"2024-08-01T04:14:07.611180+0000","last_clean":"2024-08-01T04:14:07.611180+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:14:07.611180+0000","last_undegraded":"2024-08-01T04:14:07.611180+0000","last_fullsized":"2024-08-01T04:14:07.611180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61329,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":137,"num_read_kb":131,"num_write":159,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":183987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211962,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":450296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":1084367,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841565,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":450532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":450452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1042771,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":449921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1042065,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":450748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":450232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":508393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":506653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:14:32.060 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:14:32.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:31 smithi160 ceph-mon[37224]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-08-01T04:14:32.548 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:32.978 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:32 smithi003 ceph-mon[32262]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:14:33.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:32 smithi163 ceph-mon[37383]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:14:33.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:32 smithi160 ceph-mon[37224]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 -- 172.21.15.3:0/2834616244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4448100b50 msgr2=0x7f4448100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 --2- 172.21.15.3:0/2834616244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4448100b50 0x7f4448100f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4438009c10 tx=0x7f443802a770 comp rx=0 tx=0).stop 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 -- 172.21.15.3:0/2834616244 shutdown_connections 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 --2- 172.21.15.3:0/2834616244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448102c20 0x7f444810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 --2- 172.21.15.3:0/2834616244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44481022a0 0x7f44481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 --2- 172.21.15.3:0/2834616244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4448100b50 0x7f4448100f30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 -- 172.21.15.3:0/2834616244 >> 172.21.15.3:0/2834616244 conn(0x7f44480fc500 msgr2=0x7f44480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 -- 172.21.15.3:0/2834616244 shutdown_connections 2024-08-01T04:14:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.226+0000 7f444ea00640 1 -- 172.21.15.3:0/2834616244 wait complete. 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.227+0000 7f444ea00640 1 Processor -- start 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 -- start start 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 0x7f44480fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 0x7f44480fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4448102c20 0x7f44480fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4448110cb0 con 0x7f4448102c20 2024-08-01T04:14:33.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4448110b30 con 0x7f4448100b50 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ea00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4448110e30 con 0x7f44481022a0 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f4447e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 0x7f44480fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f4447400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 0x7f44480fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f4447400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 0x7f44480fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37354/0 (socket says 172.21.15.3:37354) 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f4447e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 0x7f44480fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50290/0 (socket says 172.21.15.3:50290) 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f4447400640 1 -- 172.21.15.3:0/3672559393 learned_addr learned my addr 172.21.15.3:0/3672559393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:33.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.228+0000 7f444ca00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4448102c20 0x7f44480fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447e00640 1 -- 172.21.15.3:0/3672559393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 msgr2=0x7f44480fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447e00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 0x7f44480fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447e00640 1 -- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4448102c20 msgr2=0x7f44480fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447e00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4448102c20 0x7f44480fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447e00640 1 -- 172.21.15.3:0/3672559393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44481a66f0 con 0x7f4448100b50 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447400640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 0x7f44480fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4447e00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 0x7f44480fc9b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4438009b30 tx=0x7f44380305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.229+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438041020 con 0x7f4448100b50 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.230+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4438003e50 con 0x7f4448100b50 2024-08-01T04:14:33.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.230+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438036040 con 0x7f4448100b50 2024-08-01T04:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.230+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44481a6980 con 0x7f4448100b50 2024-08-01T04:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.230+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481a6e80 con 0x7f4448100b50 2024-08-01T04:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.231+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4448100f30 con 0x7f4448100b50 2024-08-01T04:14:33.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.233+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4438044020 con 0x7f4448100b50 2024-08-01T04:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.234+0000 7f4444c00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f442c0a2c90 0x7f442c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.234+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f44380efb50 con 0x7f4448100b50 2024-08-01T04:14:33.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.234+0000 7f4447400640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f442c0a2c90 0x7f442c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:33.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.235+0000 7f4447400640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f442c0a2c90 0x7f442c0a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f44300096c0 tx=0x7f4430009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:33.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.238+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f443804b020 con 0x7f4448100b50 2024-08-01T04:14:33.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.429+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4448062360 con 0x7f442c0a2c90 2024-08-01T04:14:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.450+0000 7f4444c00640 1 -- 172.21.15.3:0/3672559393 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258828 (secure 0 0 0) 0x7f4448062360 con 0x7f442c0a2c90 2024-08-01T04:14:33.457 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:33.458 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:14:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.457+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f442c0a2c90 msgr2=0x7f442c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.457+0000 7f444ea00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f442c0a2c90 0x7f442c0a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f44300096c0 tx=0x7f4430009260 comp rx=0 tx=0).stop 2024-08-01T04:14:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.457+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 msgr2=0x7f44480fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.457+0000 7f444ea00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 0x7f44480fc9b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4438009b30 tx=0x7f44380305d0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 shutdown_connections 2024-08-01T04:14:33.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f442c0a2c90 0x7f442c0a5150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4448102c20 0x7f44480fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44481022a0 0x7f44480fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 --2- 172.21.15.3:0/3672559393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4448100b50 0x7f44480fc9b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 >> 172.21.15.3:0/3672559393 conn(0x7f44480fc500 msgr2=0x7f4448069f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 shutdown_connections 2024-08-01T04:14:33.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:33.458+0000 7f444ea00640 1 -- 172.21.15.3:0/3672559393 wait complete. 2024-08-01T04:14:34.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:33 smithi163 ceph-mon[37383]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-08-01T04:14:34.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:33 smithi003 ceph-mon[32262]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-08-01T04:14:34.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:33 smithi160 ceph-mon[37224]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-08-01T04:14:34.456 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-08-01T04:14:32.198482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657326,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":294,"num_read_kb":271,"num_write":345,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364724,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577100,"statfs":{"total":1151940427776,"available":1151566950400,"internally_reserved":0,"allocated":11685888,"data_stored":7352361,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54357,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":100,"num_write":29,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004758"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528883+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:13:56.528883+0000","last_peered":"2024-08-01T04:13:56.528883+0000","last_clean":"2024-08-01T04:13:56.528883+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:13:56.528883+0000","last_undegraded":"2024-08-01T04:13:56.528883+0000","last_fullsized":"2024-08-01T04:13:56.528883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.537905+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:13:56.537905+0000","last_peered":"2024-08-01T04:13:56.537905+0000","last_clean":"2024-08-01T04:13:56.537905+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:13:56.537905+0000","last_undegraded":"2024-08-01T04:13:56.537905+0000","last_fullsized":"2024-08-01T04:13:56.537905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.537852+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:13:56.537852+0000","last_peered":"2024-08-01T04:13:56.537852+0000","last_clean":"2024-08-01T04:13:56.537852+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:13:56.537852+0000","last_undegraded":"2024-08-01T04:13:56.537852+0000","last_fullsized":"2024-08-01T04:13:56.537852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.505984+0000","last_change":"2024-08-01T04:13:54.507379+0000","last_active":"2024-08-01T04:13:56.505984+0000","last_peered":"2024-08-01T04:13:56.505984+0000","last_clean":"2024-08-01T04:13:56.505984+0000","last_became_active":"2024-08-01T04:13:54.507066+0000","last_became_peered":"2024-08-01T04:13:54.507066+0000","last_unstale":"2024-08-01T04:13:56.505984+0000","last_undegraded":"2024-08-01T04:13:56.505984+0000","last_fullsized":"2024-08-01T04:13:56.505984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:54.117496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529184+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:13:56.529184+0000","last_peered":"2024-08-01T04:13:56.529184+0000","last_clean":"2024-08-01T04:13:56.529184+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:13:56.529184+0000","last_undegraded":"2024-08-01T04:13:56.529184+0000","last_fullsized":"2024-08-01T04:13:56.529184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528181+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:13:56.528181+0000","last_peered":"2024-08-01T04:13:56.528181+0000","last_clean":"2024-08-01T04:13:56.528181+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:13:56.528181+0000","last_undegraded":"2024-08-01T04:13:56.528181+0000","last_fullsized":"2024-08-01T04:13:56.528181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526976+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:13:56.526976+0000","last_peered":"2024-08-01T04:13:56.526976+0000","last_clean":"2024-08-01T04:13:56.526976+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:13:56.526976+0000","last_undegraded":"2024-08-01T04:13:56.526976+0000","last_fullsized":"2024-08-01T04:13:56.526976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.508951+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:13:56.508951+0000","last_peered":"2024-08-01T04:13:56.508951+0000","last_clean":"2024-08-01T04:13:56.508951+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:13:56.508951+0000","last_undegraded":"2024-08-01T04:13:56.508951+0000","last_fullsized":"2024-08-01T04:13:56.508951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531349+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:13:56.531349+0000","last_peered":"2024-08-01T04:13:56.531349+0000","last_clean":"2024-08-01T04:13:56.531349+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:13:56.531349+0000","last_undegraded":"2024-08-01T04:13:56.531349+0000","last_fullsized":"2024-08-01T04:13:56.531349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529013+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:13:56.529013+0000","last_peered":"2024-08-01T04:13:56.529013+0000","last_clean":"2024-08-01T04:13:56.529013+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:13:56.529013+0000","last_undegraded":"2024-08-01T04:13:56.529013+0000","last_fullsized":"2024-08-01T04:13:56.529013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528977+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:13:56.528977+0000","last_peered":"2024-08-01T04:13:56.528977+0000","last_clean":"2024-08-01T04:13:56.528977+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:13:56.528977+0000","last_undegraded":"2024-08-01T04:13:56.528977+0000","last_fullsized":"2024-08-01T04:13:56.528977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506072+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:13:56.506072+0000","last_peered":"2024-08-01T04:13:56.506072+0000","last_clean":"2024-08-01T04:13:56.506072+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:13:56.506072+0000","last_undegraded":"2024-08-01T04:13:56.506072+0000","last_fullsized":"2024-08-01T04:13:56.506072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530326+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:13:56.530326+0000","last_peered":"2024-08-01T04:13:56.530326+0000","last_clean":"2024-08-01T04:13:56.530326+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:13:56.530326+0000","last_undegraded":"2024-08-01T04:13:56.530326+0000","last_fullsized":"2024-08-01T04:13:56.530326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.536358+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:13:56.536358+0000","last_peered":"2024-08-01T04:13:56.536358+0000","last_clean":"2024-08-01T04:13:56.536358+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:13:56.536358+0000","last_undegraded":"2024-08-01T04:13:56.536358+0000","last_fullsized":"2024-08-01T04:13:56.536358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.536382+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:13:56.536382+0000","last_peered":"2024-08-01T04:13:56.536382+0000","last_clean":"2024-08-01T04:13:56.536382+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:13:56.536382+0000","last_undegraded":"2024-08-01T04:13:56.536382+0000","last_fullsized":"2024-08-01T04:13:56.536382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530428+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:13:56.530428+0000","last_peered":"2024-08-01T04:13:56.530428+0000","last_clean":"2024-08-01T04:13:56.530428+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:13:56.530428+0000","last_undegraded":"2024-08-01T04:13:56.530428+0000","last_fullsized":"2024-08-01T04:13:56.530428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529079+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:13:56.529079+0000","last_peered":"2024-08-01T04:13:56.529079+0000","last_clean":"2024-08-01T04:13:56.529079+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:13:56.529079+0000","last_undegraded":"2024-08-01T04:13:56.529079+0000","last_fullsized":"2024-08-01T04:13:56.529079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.532924+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:13:56.532924+0000","last_peered":"2024-08-01T04:13:56.532924+0000","last_clean":"2024-08-01T04:13:56.532924+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:13:56.532924+0000","last_undegraded":"2024-08-01T04:13:56.532924+0000","last_fullsized":"2024-08-01T04:13:56.532924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527186+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:13:56.527186+0000","last_peered":"2024-08-01T04:13:56.527186+0000","last_clean":"2024-08-01T04:13:56.527186+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:13:56.527186+0000","last_undegraded":"2024-08-01T04:13:56.527186+0000","last_fullsized":"2024-08-01T04:13:56.527186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528008+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:13:56.528008+0000","last_peered":"2024-08-01T04:13:56.528008+0000","last_clean":"2024-08-01T04:13:56.528008+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:13:56.528008+0000","last_undegraded":"2024-08-01T04:13:56.528008+0000","last_fullsized":"2024-08-01T04:13:56.528008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530818+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:13:56.530818+0000","last_peered":"2024-08-01T04:13:56.530818+0000","last_clean":"2024-08-01T04:13:56.530818+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:13:56.530818+0000","last_undegraded":"2024-08-01T04:13:56.530818+0000","last_fullsized":"2024-08-01T04:13:56.530818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531085+0000","last_change":"2024-08-01T04:13:54.526711+0000","last_active":"2024-08-01T04:13:56.531085+0000","last_peered":"2024-08-01T04:13:56.531085+0000","last_clean":"2024-08-01T04:13:56.531085+0000","last_became_active":"2024-08-01T04:13:54.526371+0000","last_became_peered":"2024-08-01T04:13:54.526371+0000","last_unstale":"2024-08-01T04:13:56.531085+0000","last_undegraded":"2024-08-01T04:13:56.531085+0000","last_fullsized":"2024-08-01T04:13:56.531085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:24:06.453248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.508965+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:13:56.508965+0000","last_peered":"2024-08-01T04:13:56.508965+0000","last_clean":"2024-08-01T04:13:56.508965+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:13:56.508965+0000","last_undegraded":"2024-08-01T04:13:56.508965+0000","last_fullsized":"2024-08-01T04:13:56.508965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506870+0000","last_change":"2024-08-01T04:13:54.521065+0000","last_active":"2024-08-01T04:13:56.506870+0000","last_peered":"2024-08-01T04:13:56.506870+0000","last_clean":"2024-08-01T04:13:56.506870+0000","last_became_active":"2024-08-01T04:13:54.520527+0000","last_became_peered":"2024-08-01T04:13:54.520527+0000","last_unstale":"2024-08-01T04:13:56.506870+0000","last_undegraded":"2024-08-01T04:13:56.506870+0000","last_fullsized":"2024-08-01T04:13:56.506870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:43.132148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.510396+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:13:56.510396+0000","last_peered":"2024-08-01T04:13:56.510396+0000","last_clean":"2024-08-01T04:13:56.510396+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:13:56.510396+0000","last_undegraded":"2024-08-01T04:13:56.510396+0000","last_fullsized":"2024-08-01T04:13:56.510396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527435+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:13:56.527435+0000","last_peered":"2024-08-01T04:13:56.527435+0000","last_clean":"2024-08-01T04:13:56.527435+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:13:56.527435+0000","last_undegraded":"2024-08-01T04:13:56.527435+0000","last_fullsized":"2024-08-01T04:13:56.527435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528523+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:13:56.528523+0000","last_peered":"2024-08-01T04:13:56.528523+0000","last_clean":"2024-08-01T04:13:56.528523+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:13:56.528523+0000","last_undegraded":"2024-08-01T04:13:56.528523+0000","last_fullsized":"2024-08-01T04:13:56.528523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529695+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:13:56.529695+0000","last_peered":"2024-08-01T04:13:56.529695+0000","last_clean":"2024-08-01T04:13:56.529695+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:13:56.529695+0000","last_undegraded":"2024-08-01T04:13:56.529695+0000","last_fullsized":"2024-08-01T04:13:56.529695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.537002+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:13:56.537002+0000","last_peered":"2024-08-01T04:13:56.537002+0000","last_clean":"2024-08-01T04:13:56.537002+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:13:56.537002+0000","last_undegraded":"2024-08-01T04:13:56.537002+0000","last_fullsized":"2024-08-01T04:13:56.537002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526355+0000","last_change":"2024-08-01T04:13:54.526099+0000","last_active":"2024-08-01T04:13:56.526355+0000","last_peered":"2024-08-01T04:13:56.526355+0000","last_clean":"2024-08-01T04:13:56.526355+0000","last_became_active":"2024-08-01T04:13:54.525730+0000","last_became_peered":"2024-08-01T04:13:54.525730+0000","last_unstale":"2024-08-01T04:13:56.526355+0000","last_undegraded":"2024-08-01T04:13:56.526355+0000","last_fullsized":"2024-08-01T04:13:56.526355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:17:40.338865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527049+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:13:56.527049+0000","last_peered":"2024-08-01T04:13:56.527049+0000","last_clean":"2024-08-01T04:13:56.527049+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:13:56.527049+0000","last_undegraded":"2024-08-01T04:13:56.527049+0000","last_fullsized":"2024-08-01T04:13:56.527049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526947+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:13:56.526947+0000","last_peered":"2024-08-01T04:13:56.526947+0000","last_clean":"2024-08-01T04:13:56.526947+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:13:56.526947+0000","last_undegraded":"2024-08-01T04:13:56.526947+0000","last_fullsized":"2024-08-01T04:13:56.526947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529708+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:13:56.529708+0000","last_peered":"2024-08-01T04:13:56.529708+0000","last_clean":"2024-08-01T04:13:56.529708+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:13:56.529708+0000","last_undegraded":"2024-08-01T04:13:56.529708+0000","last_fullsized":"2024-08-01T04:13:56.529708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:27.657855+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:14:27.657855+0000","last_peered":"2024-08-01T04:14:27.657855+0000","last_clean":"2024-08-01T04:14:27.657855+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:14:27.657855+0000","last_undegraded":"2024-08-01T04:14:27.657855+0000","last_fullsized":"2024-08-01T04:14:27.657855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.507073+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:13:56.507073+0000","last_peered":"2024-08-01T04:13:56.507073+0000","last_clean":"2024-08-01T04:13:56.507073+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:13:56.507073+0000","last_undegraded":"2024-08-01T04:13:56.507073+0000","last_fullsized":"2024-08-01T04:13:56.507073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.609507+0000","last_change":"2024-08-01T04:13:51.458870+0000","last_active":"2024-08-01T04:14:07.609507+0000","last_peered":"2024-08-01T04:14:07.609507+0000","last_clean":"2024-08-01T04:14:07.609507+0000","last_became_active":"2024-08-01T04:13:51.458507+0000","last_became_peered":"2024-08-01T04:13:51.458507+0000","last_unstale":"2024-08-01T04:14:07.609507+0000","last_undegraded":"2024-08-01T04:14:07.609507+0000","last_fullsized":"2024-08-01T04:14:07.609507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:21:57.547811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.611983+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:14:07.611983+0000","last_peered":"2024-08-01T04:14:07.611983+0000","last_clean":"2024-08-01T04:14:07.611983+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:14:07.611983+0000","last_undegraded":"2024-08-01T04:14:07.611983+0000","last_fullsized":"2024-08-01T04:14:07.611983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.509085+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:13:56.509085+0000","last_peered":"2024-08-01T04:13:56.509085+0000","last_clean":"2024-08-01T04:13:56.509085+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:13:56.509085+0000","last_undegraded":"2024-08-01T04:13:56.509085+0000","last_fullsized":"2024-08-01T04:13:56.509085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.507426+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:13:56.507426+0000","last_peered":"2024-08-01T04:13:56.507426+0000","last_clean":"2024-08-01T04:13:56.507426+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:13:56.507426+0000","last_undegraded":"2024-08-01T04:13:56.507426+0000","last_fullsized":"2024-08-01T04:13:56.507426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.532080+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:13:56.532080+0000","last_peered":"2024-08-01T04:13:56.532080+0000","last_clean":"2024-08-01T04:13:56.532080+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:13:56.532080+0000","last_undegraded":"2024-08-01T04:13:56.532080+0000","last_fullsized":"2024-08-01T04:13:56.532080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:30.661809+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:14:30.661809+0000","last_peered":"2024-08-01T04:14:30.661809+0000","last_clean":"2024-08-01T04:14:30.661809+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:14:30.661809+0000","last_undegraded":"2024-08-01T04:14:30.661809+0000","last_fullsized":"2024-08-01T04:14:30.661809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531153+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:13:56.531153+0000","last_peered":"2024-08-01T04:13:56.531153+0000","last_clean":"2024-08-01T04:13:56.531153+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:13:56.531153+0000","last_undegraded":"2024-08-01T04:13:56.531153+0000","last_fullsized":"2024-08-01T04:13:56.531153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.602386+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:14:06.602386+0000","last_peered":"2024-08-01T04:14:06.602386+0000","last_clean":"2024-08-01T04:14:06.602386+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:14:06.602386+0000","last_undegraded":"2024-08-01T04:14:06.602386+0000","last_fullsized":"2024-08-01T04:14:06.602386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.538542+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:13:56.538542+0000","last_peered":"2024-08-01T04:13:56.538542+0000","last_clean":"2024-08-01T04:13:56.538542+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:13:56.538542+0000","last_undegraded":"2024-08-01T04:13:56.538542+0000","last_fullsized":"2024-08-01T04:13:56.538542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.601022+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:14:06.601022+0000","last_peered":"2024-08-01T04:14:06.601022+0000","last_clean":"2024-08-01T04:14:06.601022+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:14:06.601022+0000","last_undegraded":"2024-08-01T04:14:06.601022+0000","last_fullsized":"2024-08-01T04:14:06.601022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531094+0000","last_change":"2024-08-01T04:13:54.503730+0000","last_active":"2024-08-01T04:13:56.531094+0000","last_peered":"2024-08-01T04:13:56.531094+0000","last_clean":"2024-08-01T04:13:56.531094+0000","last_became_active":"2024-08-01T04:13:54.503412+0000","last_became_peered":"2024-08-01T04:13:54.503412+0000","last_unstale":"2024-08-01T04:13:56.531094+0000","last_undegraded":"2024-08-01T04:13:56.531094+0000","last_fullsized":"2024-08-01T04:13:56.531094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:12:48.161869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":104,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:31.663584+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:14:31.663584+0000","last_peered":"2024-08-01T04:14:31.663584+0000","last_clean":"2024-08-01T04:14:31.663584+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:14:31.663584+0000","last_undegraded":"2024-08-01T04:14:31.663584+0000","last_fullsized":"2024-08-01T04:14:31.663584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62334,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":54,"num_read_kb":87,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529527+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:13:56.529527+0000","last_peered":"2024-08-01T04:13:56.529527+0000","last_clean":"2024-08-01T04:13:56.529527+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:13:56.529527+0000","last_undegraded":"2024-08-01T04:13:56.529527+0000","last_fullsized":"2024-08-01T04:13:56.529527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.606498+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:14:07.606498+0000","last_peered":"2024-08-01T04:14:07.606498+0000","last_clean":"2024-08-01T04:14:07.606498+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:14:07.606498+0000","last_undegraded":"2024-08-01T04:14:07.606498+0000","last_fullsized":"2024-08-01T04:14:07.606498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529575+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:13:56.529575+0000","last_peered":"2024-08-01T04:13:56.529575+0000","last_clean":"2024-08-01T04:13:56.529575+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:13:56.529575+0000","last_undegraded":"2024-08-01T04:13:56.529575+0000","last_fullsized":"2024-08-01T04:13:56.529575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605303+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:14:06.605303+0000","last_peered":"2024-08-01T04:14:06.605303+0000","last_clean":"2024-08-01T04:14:06.605303+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:14:06.605303+0000","last_undegraded":"2024-08-01T04:14:06.605303+0000","last_fullsized":"2024-08-01T04:14:06.605303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527463+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:13:56.527463+0000","last_peered":"2024-08-01T04:13:56.527463+0000","last_clean":"2024-08-01T04:13:56.527463+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:13:56.527463+0000","last_undegraded":"2024-08-01T04:13:56.527463+0000","last_fullsized":"2024-08-01T04:13:56.527463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:26.660306+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:14:26.660306+0000","last_peered":"2024-08-01T04:14:26.660306+0000","last_clean":"2024-08-01T04:14:26.660306+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:14:26.660306+0000","last_undegraded":"2024-08-01T04:14:26.660306+0000","last_fullsized":"2024-08-01T04:14:26.660306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526833+0000","last_change":"2024-08-01T04:13:54.509611+0000","last_active":"2024-08-01T04:13:56.526833+0000","last_peered":"2024-08-01T04:13:56.526833+0000","last_clean":"2024-08-01T04:13:56.526833+0000","last_became_active":"2024-08-01T04:13:54.509246+0000","last_became_peered":"2024-08-01T04:13:54.509246+0000","last_unstale":"2024-08-01T04:13:56.526833+0000","last_undegraded":"2024-08-01T04:13:56.526833+0000","last_fullsized":"2024-08-01T04:13:56.526833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:15:08.754404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.609796+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:14:07.609796+0000","last_peered":"2024-08-01T04:14:07.609796+0000","last_clean":"2024-08-01T04:14:07.609796+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:14:07.609796+0000","last_undegraded":"2024-08-01T04:14:07.609796+0000","last_fullsized":"2024-08-01T04:14:07.609796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506977+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:13:56.506977+0000","last_peered":"2024-08-01T04:13:56.506977+0000","last_clean":"2024-08-01T04:13:56.506977+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:13:56.506977+0000","last_undegraded":"2024-08-01T04:13:56.506977+0000","last_fullsized":"2024-08-01T04:13:56.506977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:09.599827+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:14:09.599827+0000","last_peered":"2024-08-01T04:14:09.599827+0000","last_clean":"2024-08-01T04:14:09.599827+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:14:09.599827+0000","last_undegraded":"2024-08-01T04:14:09.599827+0000","last_fullsized":"2024-08-01T04:14:09.599827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528052+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:13:56.528052+0000","last_peered":"2024-08-01T04:13:56.528052+0000","last_clean":"2024-08-01T04:13:56.528052+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:13:56.528052+0000","last_undegraded":"2024-08-01T04:13:56.528052+0000","last_fullsized":"2024-08-01T04:13:56.528052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.534081+0000","last_change":"2024-08-01T04:13:51.466500+0000","last_active":"2024-08-01T04:13:56.534081+0000","last_peered":"2024-08-01T04:13:56.534081+0000","last_clean":"2024-08-01T04:13:56.534081+0000","last_became_active":"2024-08-01T04:13:51.466069+0000","last_became_peered":"2024-08-01T04:13:51.466069+0000","last_unstale":"2024-08-01T04:13:56.534081+0000","last_undegraded":"2024-08-01T04:13:56.534081+0000","last_fullsized":"2024-08-01T04:13:56.534081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:30:03.961179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528594+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:13:56.528594+0000","last_peered":"2024-08-01T04:13:56.528594+0000","last_clean":"2024-08-01T04:13:56.528594+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:13:56.528594+0000","last_undegraded":"2024-08-01T04:13:56.528594+0000","last_fullsized":"2024-08-01T04:13:56.528594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530578+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:13:56.530578+0000","last_peered":"2024-08-01T04:13:56.530578+0000","last_clean":"2024-08-01T04:13:56.530578+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:13:56.530578+0000","last_undegraded":"2024-08-01T04:13:56.530578+0000","last_fullsized":"2024-08-01T04:13:56.530578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527814+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:13:56.527814+0000","last_peered":"2024-08-01T04:13:56.527814+0000","last_clean":"2024-08-01T04:13:56.527814+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:13:56.527814+0000","last_undegraded":"2024-08-01T04:13:56.527814+0000","last_fullsized":"2024-08-01T04:13:56.527814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.606867+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:14:07.606867+0000","last_peered":"2024-08-01T04:14:07.606867+0000","last_clean":"2024-08-01T04:14:07.606867+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:14:07.606867+0000","last_undegraded":"2024-08-01T04:14:07.606867+0000","last_fullsized":"2024-08-01T04:14:07.606867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529210+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:13:56.529210+0000","last_peered":"2024-08-01T04:13:56.529210+0000","last_clean":"2024-08-01T04:13:56.529210+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:13:56.529210+0000","last_undegraded":"2024-08-01T04:13:56.529210+0000","last_fullsized":"2024-08-01T04:13:56.529210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527936+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:13:56.527936+0000","last_peered":"2024-08-01T04:13:56.527936+0000","last_clean":"2024-08-01T04:13:56.527936+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:13:56.527936+0000","last_undegraded":"2024-08-01T04:13:56.527936+0000","last_fullsized":"2024-08-01T04:13:56.527936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:09.624388+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:14:09.624388+0000","last_peered":"2024-08-01T04:14:09.624388+0000","last_clean":"2024-08-01T04:14:09.624388+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:14:09.624388+0000","last_undegraded":"2024-08-01T04:14:09.624388+0000","last_fullsized":"2024-08-01T04:14:09.624388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.509865+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:13:56.509865+0000","last_peered":"2024-08-01T04:13:56.509865+0000","last_clean":"2024-08-01T04:13:56.509865+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:13:56.509865+0000","last_undegraded":"2024-08-01T04:13:56.509865+0000","last_fullsized":"2024-08-01T04:13:56.509865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.602542+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:14:07.602542+0000","last_peered":"2024-08-01T04:14:07.602542+0000","last_clean":"2024-08-01T04:14:07.602542+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:14:07.602542+0000","last_undegraded":"2024-08-01T04:14:07.602542+0000","last_fullsized":"2024-08-01T04:14:07.602542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529278+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:13:56.529278+0000","last_peered":"2024-08-01T04:13:56.529278+0000","last_clean":"2024-08-01T04:13:56.529278+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:13:56.529278+0000","last_undegraded":"2024-08-01T04:13:56.529278+0000","last_fullsized":"2024-08-01T04:13:56.529278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:29.665208+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:14:29.665208+0000","last_peered":"2024-08-01T04:14:29.665208+0000","last_clean":"2024-08-01T04:14:29.665208+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:14:29.665208+0000","last_undegraded":"2024-08-01T04:14:29.665208+0000","last_fullsized":"2024-08-01T04:14:29.665208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.506964+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:13:56.506964+0000","last_peered":"2024-08-01T04:13:56.506964+0000","last_clean":"2024-08-01T04:13:56.506964+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:13:56.506964+0000","last_undegraded":"2024-08-01T04:13:56.506964+0000","last_fullsized":"2024-08-01T04:13:56.506964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527343+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:13:56.527343+0000","last_peered":"2024-08-01T04:13:56.527343+0000","last_clean":"2024-08-01T04:13:56.527343+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:13:56.527343+0000","last_undegraded":"2024-08-01T04:13:56.527343+0000","last_fullsized":"2024-08-01T04:13:56.527343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.528022+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:13:56.528022+0000","last_peered":"2024-08-01T04:13:56.528022+0000","last_clean":"2024-08-01T04:13:56.528022+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:13:56.528022+0000","last_undegraded":"2024-08-01T04:13:56.528022+0000","last_fullsized":"2024-08-01T04:13:56.528022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.532665+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:13:56.532665+0000","last_peered":"2024-08-01T04:13:56.532665+0000","last_clean":"2024-08-01T04:13:56.532665+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:13:56.532665+0000","last_undegraded":"2024-08-01T04:13:56.532665+0000","last_fullsized":"2024-08-01T04:13:56.532665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.921198+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:13:56.921198+0000","last_peered":"2024-08-01T04:13:56.921198+0000","last_clean":"2024-08-01T04:13:56.921198+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:13:56.921198+0000","last_undegraded":"2024-08-01T04:13:56.921198+0000","last_fullsized":"2024-08-01T04:13:56.921198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605918+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:14:06.605918+0000","last_peered":"2024-08-01T04:14:06.605918+0000","last_clean":"2024-08-01T04:14:06.605918+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:14:06.605918+0000","last_undegraded":"2024-08-01T04:14:06.605918+0000","last_fullsized":"2024-08-01T04:14:06.605918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.529861+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:13:56.529861+0000","last_peered":"2024-08-01T04:13:56.529861+0000","last_clean":"2024-08-01T04:13:56.529861+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:13:56.529861+0000","last_undegraded":"2024-08-01T04:13:56.529861+0000","last_fullsized":"2024-08-01T04:13:56.529861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:25.530465+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:14:25.530465+0000","last_peered":"2024-08-01T04:14:25.530465+0000","last_clean":"2024-08-01T04:14:25.530465+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:14:25.530465+0000","last_undegraded":"2024-08-01T04:14:25.530465+0000","last_fullsized":"2024-08-01T04:14:25.530465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.510446+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:13:56.510446+0000","last_peered":"2024-08-01T04:13:56.510446+0000","last_clean":"2024-08-01T04:13:56.510446+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:13:56.510446+0000","last_undegraded":"2024-08-01T04:13:56.510446+0000","last_fullsized":"2024-08-01T04:13:56.510446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605467+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:14:06.605467+0000","last_peered":"2024-08-01T04:14:06.605467+0000","last_clean":"2024-08-01T04:14:06.605467+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:14:06.605467+0000","last_undegraded":"2024-08-01T04:14:06.605467+0000","last_fullsized":"2024-08-01T04:14:06.605467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.538183+0000","last_change":"2024-08-01T04:13:54.532925+0000","last_active":"2024-08-01T04:13:56.538183+0000","last_peered":"2024-08-01T04:13:56.538183+0000","last_clean":"2024-08-01T04:13:56.538183+0000","last_became_active":"2024-08-01T04:13:54.531886+0000","last_became_peered":"2024-08-01T04:13:54.531886+0000","last_unstale":"2024-08-01T04:13:56.538183+0000","last_undegraded":"2024-08-01T04:13:56.538183+0000","last_fullsized":"2024-08-01T04:13:56.538183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:39.471789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.602793+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:14:06.602793+0000","last_peered":"2024-08-01T04:14:06.602793+0000","last_clean":"2024-08-01T04:14:06.602793+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:14:06.602793+0000","last_undegraded":"2024-08-01T04:14:06.602793+0000","last_fullsized":"2024-08-01T04:14:06.602793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.531055+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:13:56.531055+0000","last_peered":"2024-08-01T04:13:56.531055+0000","last_clean":"2024-08-01T04:13:56.531055+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:13:56.531055+0000","last_undegraded":"2024-08-01T04:13:56.531055+0000","last_fullsized":"2024-08-01T04:13:56.531055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.605844+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:14:06.605844+0000","last_peered":"2024-08-01T04:14:06.605844+0000","last_clean":"2024-08-01T04:14:06.605844+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:14:06.605844+0000","last_undegraded":"2024-08-01T04:14:06.605844+0000","last_fullsized":"2024-08-01T04:14:06.605844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527722+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:13:56.527722+0000","last_peered":"2024-08-01T04:13:56.527722+0000","last_clean":"2024-08-01T04:13:56.527722+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:13:56.527722+0000","last_undegraded":"2024-08-01T04:13:56.527722+0000","last_fullsized":"2024-08-01T04:13:56.527722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.526657+0000","last_change":"2024-08-01T04:13:51.465957+0000","last_active":"2024-08-01T04:13:56.526657+0000","last_peered":"2024-08-01T04:13:56.526657+0000","last_clean":"2024-08-01T04:13:56.526657+0000","last_became_active":"2024-08-01T04:13:51.465550+0000","last_became_peered":"2024-08-01T04:13:51.465550+0000","last_unstale":"2024-08-01T04:13:56.526657+0000","last_undegraded":"2024-08-01T04:13:56.526657+0000","last_fullsized":"2024-08-01T04:13:56.526657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:32:20.552737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.509429+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:13:56.509429+0000","last_peered":"2024-08-01T04:13:56.509429+0000","last_clean":"2024-08-01T04:13:56.509429+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:13:56.509429+0000","last_undegraded":"2024-08-01T04:13:56.509429+0000","last_fullsized":"2024-08-01T04:13:56.509429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:06.602064+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:14:06.602064+0000","last_peered":"2024-08-01T04:14:06.602064+0000","last_clean":"2024-08-01T04:14:06.602064+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:14:06.602064+0000","last_undegraded":"2024-08-01T04:14:06.602064+0000","last_fullsized":"2024-08-01T04:14:06.602064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.527081+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:13:56.527081+0000","last_peered":"2024-08-01T04:13:56.527081+0000","last_clean":"2024-08-01T04:13:56.527081+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:13:56.527081+0000","last_undegraded":"2024-08-01T04:13:56.527081+0000","last_fullsized":"2024-08-01T04:13:56.527081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.507217+0000","last_change":"2024-08-01T04:13:54.503954+0000","last_active":"2024-08-01T04:13:56.507217+0000","last_peered":"2024-08-01T04:13:56.507217+0000","last_clean":"2024-08-01T04:13:56.507217+0000","last_became_active":"2024-08-01T04:13:54.503613+0000","last_became_peered":"2024-08-01T04:13:54.503613+0000","last_unstale":"2024-08-01T04:13:56.507217+0000","last_undegraded":"2024-08-01T04:13:56.507217+0000","last_fullsized":"2024-08-01T04:13:56.507217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:19:01.729326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.534411+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:13:56.534411+0000","last_peered":"2024-08-01T04:13:56.534411+0000","last_clean":"2024-08-01T04:13:56.534411+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:13:56.534411+0000","last_undegraded":"2024-08-01T04:13:56.534411+0000","last_fullsized":"2024-08-01T04:13:56.534411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.530782+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:13:56.530782+0000","last_peered":"2024-08-01T04:13:56.530782+0000","last_clean":"2024-08-01T04:13:56.530782+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:13:56.530782+0000","last_undegraded":"2024-08-01T04:13:56.530782+0000","last_fullsized":"2024-08-01T04:13:56.530782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.602238+0000","last_change":"2024-08-01T04:13:51.461259+0000","last_active":"2024-08-01T04:14:07.602238+0000","last_peered":"2024-08-01T04:14:07.602238+0000","last_clean":"2024-08-01T04:14:07.602238+0000","last_became_active":"2024-08-01T04:13:51.460884+0000","last_became_peered":"2024-08-01T04:13:51.460884+0000","last_unstale":"2024-08-01T04:14:07.602238+0000","last_undegraded":"2024-08-01T04:14:07.602238+0000","last_fullsized":"2024-08-01T04:14:07.602238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:18.454778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.538110+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:13:56.538110+0000","last_peered":"2024-08-01T04:13:56.538110+0000","last_clean":"2024-08-01T04:13:56.538110+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:13:56.538110+0000","last_undegraded":"2024-08-01T04:13:56.538110+0000","last_fullsized":"2024-08-01T04:13:56.538110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.535463+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:13:56.535463+0000","last_peered":"2024-08-01T04:13:56.535463+0000","last_clean":"2024-08-01T04:13:56.535463+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:13:56.535463+0000","last_undegraded":"2024-08-01T04:13:56.535463+0000","last_fullsized":"2024-08-01T04:13:56.535463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:13:56.510546+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:13:56.510546+0000","last_peered":"2024-08-01T04:13:56.510546+0000","last_clean":"2024-08-01T04:13:56.510546+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:13:56.510546+0000","last_undegraded":"2024-08-01T04:13:56.510546+0000","last_fullsized":"2024-08-01T04:13:56.510546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-01T04:14:07.611180+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:14:07.611180+0000","last_peered":"2024-08-01T04:14:07.611180+0000","last_clean":"2024-08-01T04:14:07.611180+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:14:07.611180+0000","last_undegraded":"2024-08-01T04:14:07.611180+0000","last_fullsized":"2024-08-01T04:14:07.611180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66724,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":162,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":189382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211962,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":450296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":1099106,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841565,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":450532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":450452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1042771,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":449921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757299,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1042065,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":450748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":450232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":508393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":506653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":451192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:14:34.462 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-01T04:14:34.463 INFO:tasks.cephadm:Setting up client nodes... 2024-08-01T04:14:34.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-01T04:14:34.969 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:35.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:34 smithi163 ceph-mon[37383]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:14:35.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:34 smithi003 ceph-mon[32262]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:14:35.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:34 smithi160 ceph-mon[37224]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 -- 172.21.15.3:0/1370709214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414102970 msgr2=0x7fa414102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 --2- 172.21.15.3:0/1370709214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414102970 0x7fa414102df0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa400009900 tx=0x7fa40002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 -- 172.21.15.3:0/1370709214 shutdown_connections 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 --2- 172.21.15.3:0/1370709214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 0x7fa414109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 --2- 172.21.15.3:0/1370709214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414102970 0x7fa414102df0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 --2- 172.21.15.3:0/1370709214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414101770 0x7fa414101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.673+0000 7fa41a600640 1 -- 172.21.15.3:0/1370709214 >> 172.21.15.3:0/1370709214 conn(0x7fa4140fcf20 msgr2=0x7fa4140ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.674+0000 7fa41a600640 1 -- 172.21.15.3:0/1370709214 shutdown_connections 2024-08-01T04:14:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.674+0000 7fa41a600640 1 -- 172.21.15.3:0/1370709214 wait complete. 2024-08-01T04:14:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.675+0000 7fa41a600640 1 Processor -- start 2024-08-01T04:14:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 -- start start 2024-08-01T04:14:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414101770 0x7fa414197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414102970 0x7fa414197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 0x7fa41419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa41410d440 con 0x7fa414101770 2024-08-01T04:14:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa41410d2c0 con 0x7fa414103330 2024-08-01T04:14:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa41a600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa41410d5c0 con 0x7fa414102970 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa419200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414101770 0x7fa414197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.676+0000 7fa413e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414102970 0x7fa414197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa413e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414102970 0x7fa414197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37362/0 (socket says 172.21.15.3:37362) 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 0x7fa41419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa413e00640 1 -- 172.21.15.3:0/1555030550 learned_addr learned my addr 172.21.15.3:0/1555030550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 -- 172.21.15.3:0/1555030550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414102970 msgr2=0x7fa414197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414102970 0x7fa414197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 -- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414101770 msgr2=0x7fa414197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414101770 0x7fa414197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 -- 172.21.15.3:0/1555030550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa414074150 con 0x7fa414103330 2024-08-01T04:14:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.677+0000 7fa419c00640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 0x7fa41419f010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009fd0 tx=0x7fa40c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.678+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00ce20 con 0x7fa414103330 2024-08-01T04:14:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.678+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40c005020 con 0x7fa414103330 2024-08-01T04:14:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.678+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa414074440 con 0x7fa414103330 2024-08-01T04:14:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.678+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c010720 con 0x7fa414103330 2024-08-01T04:14:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.678+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa414074980 con 0x7fa414103330 2024-08-01T04:14:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.679+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41402d5e0 con 0x7fa414103330 2024-08-01T04:14:35.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.681+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa40c005540 con 0x7fa414103330 2024-08-01T04:14:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.683+0000 7fa411600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3fc0a2cc0 0x7fa3fc0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.683+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa40c0cf740 con 0x7fa414103330 2024-08-01T04:14:35.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.683+0000 7fa419200640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3fc0a2cc0 0x7fa3fc0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:35.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.684+0000 7fa419200640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3fc0a2cc0 0x7fa3fc0a5180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4141027d0 tx=0x7fa408005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:35.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.687+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa40c09c5f0 con 0x7fa414103330 2024-08-01T04:14:35.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.962+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa414108740 con 0x7fa414103330 2024-08-01T04:14:35.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:35 smithi003 ceph-mon[32262]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-08-01T04:14:35.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.970+0000 7fa411600640 1 -- 172.21.15.3:0/1555030550 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fa40c09bef0 con 0x7fa414103330 2024-08-01T04:14:35.972 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-08-01T04:14:35.972 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCrC6tma/aEORAAw89KgzjVfNJNXnIEjcTDDw== 2024-08-01T04:14:35.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3fc0a2cc0 msgr2=0x7fa3fc0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:35.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3fc0a2cc0 0x7fa3fc0a5180 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4141027d0 tx=0x7fa408005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:35.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 msgr2=0x7fa41419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:35.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 0x7fa41419f010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009fd0 tx=0x7fa40c00ef10 comp rx=0 tx=0).stop 2024-08-01T04:14:35.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 shutdown_connections 2024-08-01T04:14:35.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa3fc0a2cc0 0x7fa3fc0a5180 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa414103330 0x7fa41419f010 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa414102970 0x7fa414197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 --2- 172.21.15.3:0/1555030550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa414101770 0x7fa414197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:35.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 >> 172.21.15.3:0/1555030550 conn(0x7fa4140fcf20 msgr2=0x7fa414105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:35.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 shutdown_connections 2024-08-01T04:14:35.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:35.973+0000 7fa41a600640 1 -- 172.21.15.3:0/1555030550 wait complete. 2024-08-01T04:14:36.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:35 smithi163 ceph-mon[37383]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-08-01T04:14:36.053 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:35 smithi160 ceph-mon[37224]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-08-01T04:14:36.971 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:14:36.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-01T04:14:36.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-01T04:14:37.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:36 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1555030550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-01T04:14:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:36 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-01T04:14:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:36 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-01T04:14:37.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:36 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1555030550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-01T04:14:37.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:36 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-01T04:14:37.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:36 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-01T04:14:37.098 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-01T04:14:37.098 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-01T04:14:37.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph mgr dump --format=json 2024-08-01T04:14:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:36 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1555030550' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-01T04:14:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:36 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-01T04:14:37.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:36 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-01T04:14:37.626 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:38.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:37 smithi003 ceph-mon[32262]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-08-01T04:14:38.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.277+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3397383193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec1028c0 msgr2=0x7fa4ec102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:38.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.277+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3397383193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec102ca0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009900 tx=0x7fa4dc02a480 comp rx=0 tx=0).stop 2024-08-01T04:14:38.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3397383193 shutdown_connections 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3397383193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec10bcf0 0x7fa4ec10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3397383193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 0x7fa4ec10b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3397383193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec102ca0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3397383193 >> 172.21.15.3:0/3397383193 conn(0x7fa4ec0fc340 msgr2=0x7fa4ec0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3397383193 shutdown_connections 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.278+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3397383193 wait complete. 2024-08-01T04:14:38.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 Processor -- start 2024-08-01T04:14:38.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 -- start start 2024-08-01T04:14:38.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:38.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 0x7fa4ec19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec10bcf0 0x7fa4ec19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111af0 con 0x7fa4ec10bcf0 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111970 con 0x7fa4ec1028c0 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.279+0000 7fa4f1a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111c70 con 0x7fa4ec1031e0 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4f1000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec10bcf0 0x7fa4ec19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50330/0 (socket says 172.21.15.3:50330) 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 -- 172.21.15.3:0/3983790928 learned_addr learned my addr 172.21.15.3:0/3983790928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4eb400640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 0x7fa4ec19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 -- 172.21.15.3:0/3983790928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 msgr2=0x7fa4ec19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 0x7fa4ec19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 -- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec10bcf0 msgr2=0x7fa4ec19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec10bcf0 0x7fa4ec19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4ebe00640 1 -- 172.21.15.3:0/3983790928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec19f200 con 0x7fa4ec1028c0 2024-08-01T04:14:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.280+0000 7fa4eb400640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 0x7fa4ec19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.281+0000 7fa4ebe00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec19a250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc006010 tx=0x7fa4dc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:38.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.281+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc03f030 con 0x7fa4ec1028c0 2024-08-01T04:14:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.281+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc004440 con 0x7fa4ec1028c0 2024-08-01T04:14:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.281+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec19f490 con 0x7fa4ec1028c0 2024-08-01T04:14:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.281+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc035070 con 0x7fa4ec1028c0 2024-08-01T04:14:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.281+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a6dd0 con 0x7fa4ec1028c0 2024-08-01T04:14:38.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.282+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec102ca0 con 0x7fa4ec1028c0 2024-08-01T04:14:38.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.284+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa4dc042030 con 0x7fa4ec1028c0 2024-08-01T04:14:38.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.285+0000 7fa4e8c00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa4d00a2c90 0x7fa4d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.286+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa4dc0ef3f0 con 0x7fa4ec1028c0 2024-08-01T04:14:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.286+0000 7fa4eb400640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa4d00a2c90 0x7fa4d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:38.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.286+0000 7fa4eb400640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa4d00a2c90 0x7fa4d00a5150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec10e550 tx=0x7fa4d8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:38.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.290+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa4dc0bbc50 con 0x7fa4ec1028c0 2024-08-01T04:14:38.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:37 smithi163 ceph-mon[37383]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-08-01T04:14:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:37 smithi160 ceph-mon[37224]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-08-01T04:14:38.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.538+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa4ec1a6fa0 con 0x7fa4ec1028c0 2024-08-01T04:14:38.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.544+0000 7fa4e8c00640 1 -- 172.21.15.3:0/3983790928 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7fa4dc0bff50 con 0x7fa4ec1028c0 2024-08-01T04:14:38.550 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:38.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.549+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa4d00a2c90 msgr2=0x7fa4d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:38.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.549+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa4d00a2c90 0x7fa4d00a5150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec10e550 tx=0x7fa4d8005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:38.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.549+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 msgr2=0x7fa4ec19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:38.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.549+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec19a250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc006010 tx=0x7fa4dc004040 comp rx=0 tx=0).stop 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 shutdown_connections 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa4d00a2c90 0x7fa4d00a5150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4ec10bcf0 0x7fa4ec19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4ec1031e0 0x7fa4ec19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 --2- 172.21.15.3:0/3983790928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4ec1028c0 0x7fa4ec19a250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 >> 172.21.15.3:0/3983790928 conn(0x7fa4ec0fc340 msgr2=0x7fa4ec10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 shutdown_connections 2024-08-01T04:14:38.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:38.550+0000 7fa4f1a00640 1 -- 172.21.15.3:0/3983790928 wait complete. 2024-08-01T04:14:39.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:38 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3983790928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-01T04:14:39.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:38 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3983790928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-01T04:14:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:38 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3983790928' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-01T04:14:39.441 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":1617846146},{"type":"v1","addr":"172.21.15.3:6801","nonce":1617846146}]},"active_addr":"172.21.15.3:6801/1617846146","active_change":"2024-08-01T04:06:48.053852+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1348130449}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2309230633}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":346159818}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":4015366337}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1793241969}]}]} 2024-08-01T04:14:39.447 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-01T04:14:39.447 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-01T04:14:39.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:14:39.980 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:40.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:39 smithi163 ceph-mon[37383]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-08-01T04:14:40.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:39 smithi003 ceph-mon[32262]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-08-01T04:14:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:39 smithi160 ceph-mon[37224]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-08-01T04:14:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.640+0000 7f6dc9c00640 1 -- 172.21.15.3:0/959156030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc410dd90 msgr2=0x7f6dc410e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.640+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/959156030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc410dd90 0x7f6dc410e170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009930 tx=0x7f6db402a420 comp rx=0 tx=0).stop 2024-08-01T04:14:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.640+0000 7f6dc9c00640 1 -- 172.21.15.3:0/959156030 shutdown_connections 2024-08-01T04:14:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.640+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/959156030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc41086b0 0x7f6dc4114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.640+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/959156030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc4107d50 0x7f6dc4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.640+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/959156030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc410dd90 0x7f6dc410e170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.641+0000 7f6dc9c00640 1 -- 172.21.15.3:0/959156030 >> 172.21.15.3:0/959156030 conn(0x7f6dc40776b0 msgr2=0x7f6dc4079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.641+0000 7f6dc9c00640 1 -- 172.21.15.3:0/959156030 shutdown_connections 2024-08-01T04:14:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.641+0000 7f6dc9c00640 1 -- 172.21.15.3:0/959156030 wait complete. 2024-08-01T04:14:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.642+0000 7f6dc9c00640 1 Processor -- start 2024-08-01T04:14:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.642+0000 7f6dc9c00640 1 -- start start 2024-08-01T04:14:40.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc4107d50 0x7f6dc419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc41086b0 0x7f6dc419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 0x7f6dc41a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4118110 con 0x7f6dc4107d50 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4117f90 con 0x7f6dc41086b0 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4118290 con 0x7f6dc410dd90 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 0x7f6dc41a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 0x7f6dc41a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37390/0 (socket says 172.21.15.3:37390) 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc9200640 1 -- 172.21.15.3:0/3804912418 learned_addr learned my addr 172.21.15.3:0/3804912418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc3400640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc41086b0 0x7f6dc419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.643+0000 7f6dc3e00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc4107d50 0x7f6dc419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc9200640 1 -- 172.21.15.3:0/3804912418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc41086b0 msgr2=0x7f6dc419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc9200640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc41086b0 0x7f6dc419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc9200640 1 -- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc4107d50 msgr2=0x7f6dc419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc9200640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc4107d50 0x7f6dc419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc9200640 1 -- 172.21.15.3:0/3804912418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc41a3560 con 0x7f6dc410dd90 2024-08-01T04:14:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc3400640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc41086b0 0x7f6dc419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.644+0000 7f6dc9200640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 0x7f6dc41a2e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00ea60 tx=0x7f6dbc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.645+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00ce40 con 0x7f6dc410dd90 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.645+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41a3850 con 0x7f6dc410dd90 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.645+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dbc005020 con 0x7f6dc410dd90 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.645+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc010690 con 0x7f6dc410dd90 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.645+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41ab130 con 0x7f6dc410dd90 2024-08-01T04:14:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.646+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc419f910 con 0x7f6dc410dd90 2024-08-01T04:14:40.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.648+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6dbc0051d0 con 0x7f6dc410dd90 2024-08-01T04:14:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.649+0000 7f6dc0c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6da80a2c90 0x7f6da80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.649+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6dbc0cf650 con 0x7f6dc410dd90 2024-08-01T04:14:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.650+0000 7f6dc3e00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6da80a2c90 0x7f6da80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:40.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.650+0000 7f6dc3e00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6da80a2c90 0x7f6da80a5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006010 tx=0x7f6db4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:40.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.653+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6dbc09c500 con 0x7f6dc410dd90 2024-08-01T04:14:40.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.852+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6dc402d5e0 con 0x7f6dc410dd90 2024-08-01T04:14:40.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.854+0000 7f6dc0c00640 1 -- 172.21.15.3:0/3804912418 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22186 (secure 0 0 0) 0x7f6dbc09be00 con 0x7f6dc410dd90 2024-08-01T04:14:40.857 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:40.857 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":86,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:13:55.770103+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:14:40.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6da80a2c90 msgr2=0x7f6da80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:40.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6da80a2c90 0x7f6da80a5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006010 tx=0x7f6db4000bc0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 msgr2=0x7f6dc41a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 0x7f6dc41a2e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00ea60 tx=0x7f6dbc00ef20 comp rx=0 tx=0).stop 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 shutdown_connections 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6da80a2c90 0x7f6da80a5150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc410dd90 0x7f6dc41a2e80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6dc41086b0 0x7f6dc419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.857+0000 7f6dc9c00640 1 --2- 172.21.15.3:0/3804912418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dc4107d50 0x7f6dc419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.858+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 >> 172.21.15.3:0/3804912418 conn(0x7f6dc40776b0 msgr2=0x7f6dc4078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.858+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 shutdown_connections 2024-08-01T04:14:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:40.858+0000 7f6dc9c00640 1 -- 172.21.15.3:0/3804912418 wait complete. 2024-08-01T04:14:41.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:41 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3804912418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:41.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:40 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3804912418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:40 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3804912418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:41.775 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-01T04:14:41.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:14:42.285 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:42.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:42 smithi163 ceph-mon[37383]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-08-01T04:14:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:42 smithi003 ceph-mon[32262]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-08-01T04:14:42.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:42 smithi160 ceph-mon[37224]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-08-01T04:14:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 -- 172.21.15.3:0/3744795117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 msgr2=0x7f55f0103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 --2- 172.21.15.3:0/3744795117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f0103f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009930 tx=0x7f55dc02a440 comp rx=0 tx=0).stop 2024-08-01T04:14:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 -- 172.21.15.3:0/3744795117 shutdown_connections 2024-08-01T04:14:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 --2- 172.21.15.3:0/3744795117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0104470 0x7f55f01109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:42.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 --2- 172.21.15.3:0/3744795117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f0103f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 --2- 172.21.15.3:0/3744795117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0109b60 0x7f55f0109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 -- 172.21.15.3:0/3744795117 >> 172.21.15.3:0/3744795117 conn(0x7f55f00ff750 msgr2=0x7f55f0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 -- 172.21.15.3:0/3744795117 shutdown_connections 2024-08-01T04:14:42.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.961+0000 7f55f8600640 1 -- 172.21.15.3:0/3744795117 wait complete. 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.962+0000 7f55f8600640 1 Processor -- start 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.962+0000 7f55f8600640 1 -- start start 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f8600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f8600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0104470 0x7f55f019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f8600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0109b60 0x7f55f019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f8600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0113ed0 con 0x7f55f0103ad0 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f8600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0113d50 con 0x7f55f0109b60 2024-08-01T04:14:42.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f8600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0114050 con 0x7f55f0104470 2024-08-01T04:14:42.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:42.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f5200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0104470 0x7f55f019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:42.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f5c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56274/0 (socket says 172.21.15.3:56274) 2024-08-01T04:14:42.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f5c00640 1 -- 172.21.15.3:0/3486468924 learned_addr learned my addr 172.21.15.3:0/3486468924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:42.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.963+0000 7f55f6600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0109b60 0x7f55f019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5c00640 1 -- 172.21.15.3:0/3486468924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0104470 msgr2=0x7f55f019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5c00640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0104470 0x7f55f019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5c00640 1 -- 172.21.15.3:0/3486468924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0109b60 msgr2=0x7f55f019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5c00640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0109b60 0x7f55f019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5c00640 1 -- 172.21.15.3:0/3486468924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f019f3c0 con 0x7f55f0103ad0 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f6600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0109b60 0x7f55f019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5200640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0104470 0x7f55f019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f5c00640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f019a410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f55e400e9f0 tx=0x7f55e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e400cd80 con 0x7f55f0103ad0 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e4005500 con 0x7f55f0103ad0 2024-08-01T04:14:42.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4010690 con 0x7f55f0103ad0 2024-08-01T04:14:42.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.964+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f019f6b0 con 0x7f55f0103ad0 2024-08-01T04:14:42.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.965+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f01a6f90 con 0x7f55f0103ad0 2024-08-01T04:14:42.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.966+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f0103f30 con 0x7f55f0103ad0 2024-08-01T04:14:42.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.968+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f55e40046c0 con 0x7f55f0103ad0 2024-08-01T04:14:42.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.970+0000 7f55ee000640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f55d80a2c90 0x7f55d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:42.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.970+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55e40cf520 con 0x7f55f0103ad0 2024-08-01T04:14:42.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.970+0000 7f55f5200640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f55d80a2c90 0x7f55d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:42.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.971+0000 7f55f5200640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f55d80a2c90 0x7f55d80a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:42.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:42.974+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f55e409c3d0 con 0x7f55f0103ad0 2024-08-01T04:14:43.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.173+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55f010bab0 con 0x7f55f0103ad0 2024-08-01T04:14:43.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.178+0000 7f55ee000640 1 -- 172.21.15.3:0/3486468924 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22186 (secure 0 0 0) 0x7f55e409bcd0 con 0x7f55f0103ad0 2024-08-01T04:14:43.180 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:14:43.181 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":86,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:13:55.770103+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:14:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f55d80a2c90 msgr2=0x7f55d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f55d80a2c90 0x7f55d80a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc0058d0 comp rx=0 tx=0).stop 2024-08-01T04:14:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 msgr2=0x7f55f019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f019a410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f55e400e9f0 tx=0x7f55e400edf0 comp rx=0 tx=0).stop 2024-08-01T04:14:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 shutdown_connections 2024-08-01T04:14:43.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f55d80a2c90 0x7f55d80a5150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f0109b60 0x7f55f019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55f0104470 0x7f55f019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 --2- 172.21.15.3:0/3486468924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f0103ad0 0x7f55f019a410 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 >> 172.21.15.3:0/3486468924 conn(0x7f55f00ff750 msgr2=0x7f55f01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 shutdown_connections 2024-08-01T04:14:43.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:43.181+0000 7f55f8600640 1 -- 172.21.15.3:0/3486468924 wait complete. 2024-08-01T04:14:44.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-01T04:14:44.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-01T04:14:44.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-01T04:14:44.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-01T04:14:44.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-01T04:14:44.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-01T04:14:44.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-01T04:14:44.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-01T04:14:44.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-01T04:14:44.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-01T04:14:44.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-01T04:14:44.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-01T04:14:44.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:44 smithi003 ceph-mon[32262]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-01T04:14:44.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:44 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3486468924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:44.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:44 smithi163 ceph-mon[37383]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-01T04:14:44.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:44 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3486468924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:44.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:44 smithi160 ceph-mon[37224]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-01T04:14:44.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:44 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3486468924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:14:45.073 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:45.308 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:46.179 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:46 smithi003 ceph-mon[32262]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-01T04:14:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 -- 172.21.15.3:0/936080927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801078a0 msgr2=0x7f2280107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 --2- 172.21.15.3:0/936080927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801078a0 0x7f2280107c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2274009930 tx=0x7f227402a420 comp rx=0 tx=0).stop 2024-08-01T04:14:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 -- 172.21.15.3:0/936080927 shutdown_connections 2024-08-01T04:14:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 --2- 172.21.15.3:0/936080927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2280102240 0x7f228010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 --2- 172.21.15.3:0/936080927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22801018a0 0x7f2280101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 --2- 172.21.15.3:0/936080927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801078a0 0x7f2280107c80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.135+0000 7f2287600640 1 -- 172.21.15.3:0/936080927 >> 172.21.15.3:0/936080927 conn(0x7f22800fd5c0 msgr2=0x7f22800ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.136+0000 7f2287600640 1 -- 172.21.15.3:0/936080927 shutdown_connections 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.136+0000 7f2287600640 1 -- 172.21.15.3:0/936080927 wait complete. 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 Processor -- start 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 -- start start 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801018a0 0x7f228019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2280102240 0x7f228019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22801078a0 0x7f228019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111cf0 con 0x7f22801018a0 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111b70 con 0x7f2280102240 2024-08-01T04:14:46.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.137+0000 7f2287600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111e70 con 0x7f22801078a0 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2284c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801018a0 0x7f228019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2284c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801018a0 0x7f228019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56300/0 (socket says 172.21.15.3:56300) 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2284c00640 1 -- 172.21.15.3:0/1912134822 learned_addr learned my addr 172.21.15.3:0/1912134822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2285600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22801078a0 0x7f228019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f227fe00640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2280102240 0x7f228019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2285600640 1 -- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2280102240 msgr2=0x7f228019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2285600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2280102240 0x7f228019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2285600640 1 -- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801018a0 msgr2=0x7f228019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2285600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801018a0 0x7f228019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2285600640 1 -- 172.21.15.3:0/1912134822 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228019f3e0 con 0x7f22801078a0 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.138+0000 7f2284c00640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f22801018a0 0x7f228019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:46.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.139+0000 7f2285600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22801078a0 0x7f228019ed00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2278009fd0 tx=0x7f227800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.139+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227800ce20 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.139+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2278005020 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.139+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278010720 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.139+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228019f6d0 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.139+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a6fb0 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.140+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22800fef90 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.142+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2278005540 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.145+0000 7f227d600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f22680a2c90 0x7f22680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.145+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f22780cf850 con 0x7f22801078a0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.145+0000 7f2284c00640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f22680a2c90 0x7f22680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.145+0000 7f227d600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] conn(0x7f22680ac4e0 0x7f22680ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.146+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 --> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22680aefb0 con 0x7f22680ac4e0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.146+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f22780cfc00 con 0x7f22801078a0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.146+0000 7f227fe00640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] conn(0x7f22680ac4e0 0x7f22680ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.146+0000 7f2284c00640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f22680a2c90 0x7f22680a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2274000c00 tx=0x7f2274031510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.147+0000 7f227fe00640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] conn(0x7f22680ac4e0 0x7f22680ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.147+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== osd.1 v2:172.21.15.160:6800/1858231238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22680aefb0 con 0x7f22680ac4e0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.162+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 --> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2280102fe0 con 0x7f22680ac4e0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.163+0000 7f227d600640 1 -- 172.21.15.3:0/1912134822 <== osd.1 v2:172.21.15.160:6800/1858231238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2280102fe0 con 0x7f22680ac4e0 2024-08-01T04:14:46.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.163+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] conn(0x7f22680ac4e0 msgr2=0x7f22680ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.163+0000 7f2287600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] conn(0x7f22680ac4e0 0x7f22680ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f22680a2c90 msgr2=0x7f22680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f22680a2c90 0x7f22680a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2274000c00 tx=0x7f2274031510 comp rx=0 tx=0).stop 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22801078a0 msgr2=0x7f228019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 --2- 172.21.15.3:0/1912134822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22801078a0 0x7f228019ed00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2278009fd0 tx=0x7f227800ef10 comp rx=0 tx=0).stop 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2285600640 1 -- 172.21.15.3:0/1912134822 reap_dead start 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 shutdown_connections 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 >> 172.21.15.3:0/1912134822 conn(0x7f22800fd5c0 msgr2=0x7f228010ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 shutdown_connections 2024-08-01T04:14:46.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.164+0000 7f2287600640 1 -- 172.21.15.3:0/1912134822 wait complete. 2024-08-01T04:14:46.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:46 smithi163 ceph-mon[37383]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-01T04:14:46.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:46 smithi160 ceph-mon[37224]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-01T04:14:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 -- 172.21.15.3:0/175940627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 msgr2=0x7f5ecc106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 --2- 172.21.15.3:0/175940627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 0x7f5ecc106c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009890 tx=0x7f5ebc02a360 comp rx=0 tx=0).stop 2024-08-01T04:14:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 -- 172.21.15.3:0/175940627 shutdown_connections 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 --2- 172.21.15.3:0/175940627 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc100230 0x7f5ecc10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 --2- 172.21.15.3:0/175940627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc0ff890 0x7f5ecc0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 --2- 172.21.15.3:0/175940627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 0x7f5ecc106c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 -- 172.21.15.3:0/175940627 >> 172.21.15.3:0/175940627 conn(0x7f5ecc0fb7c0 msgr2=0x7f5ecc0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.221+0000 7f5ed2800640 1 -- 172.21.15.3:0/175940627 shutdown_connections 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.222+0000 7f5ed2800640 1 -- 172.21.15.3:0/175940627 wait complete. 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.222+0000 7f5ed2800640 1 Processor -- start 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 -- start start 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff890 0x7f5ecc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc100230 0x7f5ecc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 0x7f5ecc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111c90 con 0x7f5ecc100230 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111b10 con 0x7f5ecc0ff890 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ed2800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111e10 con 0x7f5ecc1068b0 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ecb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff890 0x7f5ecc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ecb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff890 0x7f5ecc19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:58354/0 (socket says 172.21.15.3:58354) 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ecb400640 1 -- 172.21.15.3:0/850244278 learned_addr learned my addr 172.21.15.3:0/850244278 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ec3e00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc100230 0x7f5ecc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.223+0000 7f5ecbe00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 0x7f5ecc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ecb400640 1 -- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 msgr2=0x7f5ecc19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ecb400640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ecc1068b0 0x7f5ecc19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ecb400640 1 -- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc100230 msgr2=0x7f5ecc19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ecb400640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ecc100230 0x7f5ecc19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ecb400640 1 -- 172.21.15.3:0/850244278 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc19f360 con 0x7f5ecc0ff890 2024-08-01T04:14:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ecb400640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff890 0x7f5ecc19a3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc033ae0 tx=0x7f5ebc033b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc037040 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc19f5f0 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ebc040070 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.224+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc003960 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.225+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a6f30 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.225+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ecc0fd200 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.227+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5ebc030580 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ec8c00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5ebc0ef9f0 con 0x7f5ecc0ff890 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ec3e00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ec8c00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] conn(0x7f5eb00ac460 0x7f5eb00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 --> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5eb00aef30 con 0x7f5eb00ac460 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc0efda0 con 0x7f5ecc0ff890 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.229+0000 7f5ecbe00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] conn(0x7f5eb00ac460 0x7f5eb00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.230+0000 7f5ec3e00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc10e960 tx=0x7f5eb4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.230+0000 7f5ecbe00640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] conn(0x7f5eb00ac460 0x7f5eb00ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.231+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== osd.8 v2:172.21.15.163:6816/2678360734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5eb00aef30 con 0x7f5eb00ac460 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.246+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 --> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ecc100db0 con 0x7f5eb00ac460 2024-08-01T04:14:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.247+0000 7f5ec8c00640 1 -- 172.21.15.3:0/850244278 <== osd.8 v2:172.21.15.163:6816/2678360734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ecc100db0 con 0x7f5eb00ac460 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] conn(0x7f5eb00ac460 msgr2=0x7f5eb00ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] conn(0x7f5eb00ac460 0x7f5eb00ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5eb00a2c10 msgr2=0x7f5eb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f5eb00a2c10 0x7f5eb00a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc10e960 tx=0x7f5eb4005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff890 msgr2=0x7f5ecc19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 --2- 172.21.15.3:0/850244278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff890 0x7f5ecc19a3b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc033ae0 tx=0x7f5ebc033b10 comp rx=0 tx=0).stop 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ecbe00640 1 -- 172.21.15.3:0/850244278 reap_dead start 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 shutdown_connections 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 >> 172.21.15.3:0/850244278 conn(0x7f5ecc0fb7c0 msgr2=0x7f5ecc10cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 shutdown_connections 2024-08-01T04:14:46.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:46.248+0000 7f5ed2800640 1 -- 172.21.15.3:0/850244278 wait complete. 2024-08-01T04:14:46.960 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.123 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.124 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.125 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.126 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.132 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.137 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.138 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.147 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.147 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:47.536 INFO:teuthology.orchestra.run.smithi003.stdout:249108103209 2024-08-01T04:14:47.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-01T04:14:47.944 INFO:teuthology.orchestra.run.smithi003.stdout:55834574923 2024-08-01T04:14:47.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-01T04:14:48.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:48 smithi003 ceph-mon[32262]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:14:48.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:48 smithi163 ceph-mon[37383]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:14:48.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:48 smithi160 ceph-mon[37224]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.800+0000 7fdcb7200640 1 -- 172.21.15.3:0/2487285489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb810df90 msgr2=0x7fdcb8110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.800+0000 7fdcb7200640 1 --2- 172.21.15.3:0/2487285489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb810df90 0x7fdcb8110380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009900 tx=0x7fdca802a480 comp rx=0 tx=0).stop 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.800+0000 7fdcb7200640 1 -- 172.21.15.3:0/2487285489 shutdown_connections 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.800+0000 7fdcb7200640 1 --2- 172.21.15.3:0/2487285489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb810df90 0x7fdcb8110380 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.800+0000 7fdcb7200640 1 --2- 172.21.15.3:0/2487285489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074c10 0x7fdcb810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.800+0000 7fdcb7200640 1 --2- 172.21.15.3:0/2487285489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb80742f0 0x7fdcb80746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.801+0000 7fdcb7200640 1 -- 172.21.15.3:0/2487285489 >> 172.21.15.3:0/2487285489 conn(0x7fdcb80fd470 msgr2=0x7fdcb80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.801+0000 7fdcb7200640 1 -- 172.21.15.3:0/2487285489 shutdown_connections 2024-08-01T04:14:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.801+0000 7fdcb7200640 1 -- 172.21.15.3:0/2487285489 wait complete. 2024-08-01T04:14:48.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 Processor -- start 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 -- start start 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb80742f0 0x7fdcb819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074c10 0x7fdcb819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb810df90 0x7fdcb819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8113d20 con 0x7fdcb80742f0 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8113ba0 con 0x7fdcb810df90 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.802+0000 7fdcb7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8113ea0 con 0x7fdcb8074c10 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb80742f0 0x7fdcb819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb80742f0 0x7fdcb819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56350/0 (socket says 172.21.15.3:56350) 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 -- 172.21.15.3:0/1564229815 learned_addr learned my addr 172.21.15.3:0/1564229815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5400640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074c10 0x7fdcb819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:48.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb6800640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb810df90 0x7fdcb819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:48.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 -- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074c10 msgr2=0x7fdcb819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8074c10 0x7fdcb819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 -- 172.21.15.3:0/1564229815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb810df90 msgr2=0x7fdcb819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb810df90 0x7fdcb819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb5e00640 1 -- 172.21.15.3:0/1564229815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb819f140 con 0x7fdcb80742f0 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.803+0000 7fdcb6800640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdcb810df90 0x7fdcb819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.804+0000 7fdcb5e00640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb80742f0 0x7fdcb819a190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ebc0 tx=0x7fdcb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.804+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0010630 con 0x7fdcb80742f0 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.804+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0010cb0 con 0x7fdcb80742f0 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.804+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0018bd0 con 0x7fdcb80742f0 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.804+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb819f430 con 0x7fdcb80742f0 2024-08-01T04:14:48.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.804+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb81a6d10 con 0x7fdcb80742f0 2024-08-01T04:14:48.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.805+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcb80ff510 con 0x7fdcb80742f0 2024-08-01T04:14:48.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.807+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdcb00107d0 con 0x7fdcb80742f0 2024-08-01T04:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.809+0000 7fdcaea00640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdc980a2c90 0x7fdc980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.809+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdcb00cfb00 con 0x7fdcb80742f0 2024-08-01T04:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.809+0000 7fdcb5400640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdc980a2c90 0x7fdc980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.809+0000 7fdcaea00640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] conn(0x7fdc980ac4e0 0x7fdc980ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.809+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 --> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc980aefb0 con 0x7fdc980ac4e0 2024-08-01T04:14:48.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.809+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fdcb00cfeb0 con 0x7fdcb80742f0 2024-08-01T04:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.810+0000 7fdcb6800640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] conn(0x7fdc980ac4e0 0x7fdc980ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.810+0000 7fdcb5400640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdc980a2c90 0x7fdc980a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80ffa00 tx=0x7fdca0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:48.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.810+0000 7fdcb6800640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] conn(0x7fdc980ac4e0 0x7fdc980ae900 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:48.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.811+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== osd.11 v2:172.21.15.163:6824/2977436589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc980aefb0 con 0x7fdc980ac4e0 2024-08-01T04:14:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.826+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 --> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcb80759a0 con 0x7fdc980ac4e0 2024-08-01T04:14:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.827+0000 7fdcaea00640 1 -- 172.21.15.3:0/1564229815 <== osd.11 v2:172.21.15.163:6824/2977436589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdcb80759a0 con 0x7fdc980ac4e0 2024-08-01T04:14:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.827+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] conn(0x7fdc980ac4e0 msgr2=0x7fdc980ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.827+0000 7fdcb7200640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] conn(0x7fdc980ac4e0 0x7fdc980ae900 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdc980a2c90 msgr2=0x7fdc980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fdc980a2c90 0x7fdc980a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80ffa00 tx=0x7fdca0005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb80742f0 msgr2=0x7fdcb819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 --2- 172.21.15.3:0/1564229815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcb80742f0 0x7fdcb819a190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000ebc0 tx=0x7fdcb000c6a0 comp rx=0 tx=0).stop 2024-08-01T04:14:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb6800640 1 -- 172.21.15.3:0/1564229815 reap_dead start 2024-08-01T04:14:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 shutdown_connections 2024-08-01T04:14:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 >> 172.21.15.3:0/1564229815 conn(0x7fdcb80fd470 msgr2=0x7fdcb80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 shutdown_connections 2024-08-01T04:14:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:48.828+0000 7fdcb7200640 1 -- 172.21.15.3:0/1564229815 wait complete. 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]: dispatch 2024-08-01T04:14:49.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 -- 172.21.15.3:0/403793129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72410dd90 msgr2=0x7fe72410e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 --2- 172.21.15.3:0/403793129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72410dd90 0x7fe72410e170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe718009930 tx=0x7fe71802a420 comp rx=0 tx=0).stop 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 -- 172.21.15.3:0/403793129 shutdown_connections 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 --2- 172.21.15.3:0/403793129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7241086b0 0x7fe724114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 --2- 172.21.15.3:0/403793129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe724107d50 0x7fe724108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 --2- 172.21.15.3:0/403793129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe72410dd90 0x7fe72410e170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 -- 172.21.15.3:0/403793129 >> 172.21.15.3:0/403793129 conn(0x7fe7240776b0 msgr2=0x7fe724079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 -- 172.21.15.3:0/403793129 shutdown_connections 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.038+0000 7fe72a600640 1 -- 172.21.15.3:0/403793129 wait complete. 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.039+0000 7fe72a600640 1 Processor -- start 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.039+0000 7fe72a600640 1 -- start start 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe72a600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe724107d50 0x7fe72419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe72a600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7241086b0 0x7fe72419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe72a600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72410dd90 0x7fe7241a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe72a600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe724118160 con 0x7fe724107d50 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe72a600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe724117fe0 con 0x7fe72410dd90 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe72a600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe7241182e0 con 0x7fe7241086b0 2024-08-01T04:14:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe729200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe724107d50 0x7fe72419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe729200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe724107d50 0x7fe72419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56354/0 (socket says 172.21.15.3:56354) 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe723e00640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7241086b0 0x7fe72419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe729200640 1 -- 172.21.15.3:0/1679616535 learned_addr learned my addr 172.21.15.3:0/1679616535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.040+0000 7fe729c00640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72410dd90 0x7fe7241a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.041+0000 7fe729200640 1 -- 172.21.15.3:0/1679616535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7241086b0 msgr2=0x7fe72419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.041+0000 7fe729200640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7241086b0 0x7fe72419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.041+0000 7fe729200640 1 -- 172.21.15.3:0/1679616535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72410dd90 msgr2=0x7fe7241a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.041+0000 7fe729200640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72410dd90 0x7fe7241a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.041+0000 7fe729200640 1 -- 172.21.15.3:0/1679616535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a35d0 con 0x7fe724107d50 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.041+0000 7fe729200640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe724107d50 0x7fe72419e620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe71802a940 tx=0x7fe718003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.042+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71803f070 con 0x7fe724107d50 2024-08-01T04:14:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.042+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a3860 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.042+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe718031c90 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.042+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718035030 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.042+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241ab1a0 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.043+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7240626b0 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.045+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe718042070 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.046+0000 7fe721600640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe70c0a2c10 0x7fe70c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.046+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe7180ef960 con 0x7fe724107d50 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.047+0000 7fe723e00640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe70c0a2c10 0x7fe70c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.047+0000 7fe721600640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] conn(0x7fe70c0ac8c0 0x7fe70c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.047+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 --> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe71802fec0 con 0x7fe70c0ac8c0 2024-08-01T04:14:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.047+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe7180f42a0 con 0x7fe724107d50 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.047+0000 7fe729c00640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] conn(0x7fe70c0ac8c0 0x7fe70c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.047+0000 7fe723e00640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe70c0a2c10 0x7fe70c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe72410d490 tx=0x7fe71000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.048+0000 7fe729c00640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] conn(0x7fe70c0ac8c0 0x7fe70c0aed60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.049+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== osd.9 v2:172.21.15.3:6826/2204833975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe71802fec0 con 0x7fe70c0ac8c0 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.064+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 --> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe724062b30 con 0x7fe70c0ac8c0 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.065+0000 7fe721600640 1 -- 172.21.15.3:0/1679616535 <== osd.9 v2:172.21.15.3:6826/2204833975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe724062b30 con 0x7fe70c0ac8c0 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.065+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] conn(0x7fe70c0ac8c0 msgr2=0x7fe70c0aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.065+0000 7fe72a600640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] conn(0x7fe70c0ac8c0 0x7fe70c0aed60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe70c0a2c10 msgr2=0x7fe70c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fe70c0a2c10 0x7fe70c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe72410d490 tx=0x7fe71000b390 comp rx=0 tx=0).stop 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe724107d50 msgr2=0x7fe72419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 --2- 172.21.15.3:0/1679616535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe724107d50 0x7fe72419e620 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe71802a940 tx=0x7fe718003b90 comp rx=0 tx=0).stop 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe729c00640 1 -- 172.21.15.3:0/1679616535 reap_dead start 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 shutdown_connections 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 >> 172.21.15.3:0/1679616535 conn(0x7fe7240776b0 msgr2=0x7fe724078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 shutdown_connections 2024-08-01T04:14:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.066+0000 7fe72a600640 1 -- 172.21.15.3:0/1679616535 wait complete. 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-08-01T04:14:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]: dispatch 2024-08-01T04:14:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-08-01T04:14:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]: dispatch 2024-08-01T04:14:49.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-08-01T04:14:49.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 -- 172.21.15.3:0/2992031726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281042a0 msgr2=0x7ff0281107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 --2- 172.21.15.3:0/2992031726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281042a0 0x7ff0281107d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff020009930 tx=0x7ff02002a420 comp rx=0 tx=0).stop 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 -- 172.21.15.3:0/2992031726 shutdown_connections 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 --2- 172.21.15.3:0/2992031726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281042a0 0x7ff0281107d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 --2- 172.21.15.3:0/2992031726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff028103900 0x7ff028103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 --2- 172.21.15.3:0/2992031726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff028109900 0x7ff028109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 -- 172.21.15.3:0/2992031726 >> 172.21.15.3:0/2992031726 conn(0x7ff0280ff600 msgr2=0x7ff028101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 -- 172.21.15.3:0/2992031726 shutdown_connections 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.885+0000 7ff02d000640 1 -- 172.21.15.3:0/2992031726 wait complete. 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 Processor -- start 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 -- start start 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff028103900 0x7ff02819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281042a0 0x7ff02819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff028109900 0x7ff02819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff028113d00 con 0x7ff028103900 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff028113b80 con 0x7ff0281042a0 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.886+0000 7ff02d000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff028113e80 con 0x7ff028109900 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff028103900 0x7ff02819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff028103900 0x7ff02819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56372/0 (socket says 172.21.15.3:56372) 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 -- 172.21.15.3:0/3258543588 learned_addr learned my addr 172.21.15.3:0/3258543588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff01ea00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281042a0 0x7ff02819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027e00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff028109900 0x7ff02819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 -- 172.21.15.3:0/3258543588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff028109900 msgr2=0x7ff02819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff028109900 0x7ff02819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 -- 172.21.15.3:0/3258543588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281042a0 msgr2=0x7ff02819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281042a0 0x7ff02819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff027400640 1 -- 172.21.15.3:0/3258543588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02819f150 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.887+0000 7ff01ea00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0281042a0 0x7ff02819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.888+0000 7ff027400640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff028103900 0x7ff02819a1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.888+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800cd80 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.888+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018005500 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.888+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02819f440 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.888+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018010690 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.888+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281a6d20 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.889+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0280626b0 con 0x7ff028103900 2024-08-01T04:14:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.891+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff0180046c0 con 0x7ff028103900 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff024c00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff00c0a2c10 0x7ff00c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff0180cfa30 con 0x7ff028103900 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff01ea00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff00c0a2c10 0x7ff00c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff024c00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] conn(0x7ff00c0ac8c0 0x7ff00c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 --> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff018004470 con 0x7ff00c0ac8c0 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ff0180cfde0 con 0x7ff028103900 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.893+0000 7ff027e00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] conn(0x7ff00c0ac8c0 0x7ff00c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.894+0000 7ff01ea00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff00c0a2c10 0x7ff00c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff028108f10 tx=0x7ff010009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.894+0000 7ff027e00640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] conn(0x7ff00c0ac8c0 0x7ff00c0aed60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:49.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.895+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== osd.0 v2:172.21.15.3:6802/1021833457 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff018004470 con 0x7ff00c0ac8c0 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.911+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 --> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff028062ab0 con 0x7ff00c0ac8c0 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.911+0000 7ff024c00640 1 -- 172.21.15.3:0/3258543588 <== osd.0 v2:172.21.15.3:6802/1021833457 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff028062ab0 con 0x7ff00c0ac8c0 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.911+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] conn(0x7ff00c0ac8c0 msgr2=0x7ff00c0aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.911+0000 7ff02d000640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] conn(0x7ff00c0ac8c0 0x7ff00c0aed60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff00c0a2c10 msgr2=0x7ff00c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff00c0a2c10 0x7ff00c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff028108f10 tx=0x7ff010009500 comp rx=0 tx=0).stop 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff028103900 msgr2=0x7ff02819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 --2- 172.21.15.3:0/3258543588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff028103900 0x7ff02819a1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800edf0 comp rx=0 tx=0).stop 2024-08-01T04:14:49.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff027e00640 1 -- 172.21.15.3:0/3258543588 reap_dead start 2024-08-01T04:14:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 shutdown_connections 2024-08-01T04:14:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 >> 172.21.15.3:0/3258543588 conn(0x7ff0280ff600 msgr2=0x7ff028100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 shutdown_connections 2024-08-01T04:14:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:49.912+0000 7ff02d000640 1 -- 172.21.15.3:0/3258543588 wait complete. 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-08-01T04:14:50.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-08-01T04:14:50.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]': finished 2024-08-01T04:14:50.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-01T04:14:50.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:50 smithi163 ceph-mon[37383]: osdmap e87: 12 total, 12 up, 12 in 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-08-01T04:14:50.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]': finished 2024-08-01T04:14:50.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-01T04:14:50.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:50 smithi003 ceph-mon[32262]: osdmap e87: 12 total, 12 up, 12 in 2024-08-01T04:14:50.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:14:50.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 6]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-01T04:14:50.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:50 smithi160 ceph-mon[37224]: osdmap e87: 12 total, 12 up, 12 in 2024-08-01T04:14:51.282 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:51 smithi003 ceph-mon[32262]: osdmap e88: 12 total, 12 up, 12 in 2024-08-01T04:14:51.283 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:51 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:14:51.286 INFO:teuthology.orchestra.run.smithi003.stdout:34359738448 2024-08-01T04:14:51.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-01T04:14:51.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:51 smithi160 ceph-mon[37224]: osdmap e88: 12 total, 12 up, 12 in 2024-08-01T04:14:51.341 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:51 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:14:51.386 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:51 smithi163 ceph-mon[37383]: osdmap e88: 12 total, 12 up, 12 in 2024-08-01T04:14:51.386 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:51 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:14:51.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.706+0000 7f75cfe00640 1 -- 172.21.15.3:0/1837732485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d01020a0 msgr2=0x7f75d010e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.706+0000 7f75cfe00640 1 --2- 172.21.15.3:0/1837732485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d01020a0 0x7f75d010e5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009ca0 tx=0x7f75c802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 -- 172.21.15.3:0/1837732485 shutdown_connections 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 --2- 172.21.15.3:0/1837732485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d01020a0 0x7f75d010e5d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 --2- 172.21.15.3:0/1837732485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 0x7f75d0101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 --2- 172.21.15.3:0/1837732485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75d0107700 0x7f75d0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 -- 172.21.15.3:0/1837732485 >> 172.21.15.3:0/1837732485 conn(0x7f75d00fd440 msgr2=0x7f75d00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 -- 172.21.15.3:0/1837732485 shutdown_connections 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.707+0000 7f75cfe00640 1 -- 172.21.15.3:0/1837732485 wait complete. 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 Processor -- start 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 -- start start 2024-08-01T04:14:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 0x7f75d019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75d01020a0 0x7f75d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d0107700 0x7f75d019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0111b50 con 0x7f75d01020a0 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01119d0 con 0x7f75d0107700 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.708+0000 7f75cfe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0111cd0 con 0x7f75d0101700 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 0x7f75d019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 0x7f75d019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55006/0 (socket says 172.21.15.3:55006) 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 -- 172.21.15.3:0/4134965641 learned_addr learned my addr 172.21.15.3:0/4134965641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cf400640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d0107700 0x7f75d019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75ce000640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75d01020a0 0x7f75d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 -- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d0107700 msgr2=0x7f75d019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d0107700 0x7f75d019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 -- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75d01020a0 msgr2=0x7f75d019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75d01020a0 0x7f75d019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 -- 172.21.15.3:0/4134965641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d019f120 con 0x7f75d0101700 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cf400640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75d0107700 0x7f75d019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.709+0000 7f75cea00640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 0x7f75d019a170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75c000ea60 tx=0x7f75c000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.710+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000cde0 con 0x7f75d0101700 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.710+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c00035f0 con 0x7f75d0101700 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.710+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0010670 con 0x7f75d0101700 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.710+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d019f410 con 0x7f75d0101700 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.710+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a6cf0 con 0x7f75d0101700 2024-08-01T04:14:51.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.711+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75d00ff3a0 con 0x7f75d0101700 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.713+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f75c00037a0 con 0x7f75d0101700 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.715+0000 7f75c7400640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f75b40a2c90 0x7f75b40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.715+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f75c00cf740 con 0x7f75d0101700 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.715+0000 7f75ce000640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f75b40a2c90 0x7f75b40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.715+0000 7f75c7400640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] conn(0x7f75b40ac940 0x7f75b40aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.715+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 --> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75c0023e50 con 0x7f75b40ac940 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.715+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f75c00cfaf0 con 0x7f75d0101700 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.716+0000 7f75cf400640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] conn(0x7f75b40ac940 0x7f75b40aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:51.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.716+0000 7f75ce000640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f75b40a2c90 0x7f75b40a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75d010e250 tx=0x7f75bc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.716+0000 7f75cf400640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] conn(0x7f75b40ac940 0x7f75b40aede0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.717+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== osd.4 v2:172.21.15.160:6808/1218975759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75c0023e50 con 0x7f75b40ac940 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.733+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 --> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75d0102e40 con 0x7f75b40ac940 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.733+0000 7f75c7400640 1 -- 172.21.15.3:0/4134965641 <== osd.4 v2:172.21.15.160:6808/1218975759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f75d0102e40 con 0x7f75b40ac940 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] conn(0x7f75b40ac940 msgr2=0x7f75b40aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] conn(0x7f75b40ac940 0x7f75b40aede0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f75b40a2c90 msgr2=0x7f75b40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f75b40a2c90 0x7f75b40a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75d010e250 tx=0x7f75bc005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 msgr2=0x7f75d019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:51.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 --2- 172.21.15.3:0/4134965641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75d0101700 0x7f75d019a170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75c000ea60 tx=0x7f75c000ef20 comp rx=0 tx=0).stop 2024-08-01T04:14:51.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cf400640 1 -- 172.21.15.3:0/4134965641 reap_dead start 2024-08-01T04:14:51.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 shutdown_connections 2024-08-01T04:14:51.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 >> 172.21.15.3:0/4134965641 conn(0x7f75d00fd440 msgr2=0x7f75d00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:51.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 shutdown_connections 2024-08-01T04:14:51.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.734+0000 7f75cfe00640 1 -- 172.21.15.3:0/4134965641 wait complete. 2024-08-01T04:14:52.309 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:52 smithi003 ceph-mon[32262]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-08-01T04:14:52.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:52 smithi160 ceph-mon[37224]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-08-01T04:14:52.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 -- 172.21.15.3:0/3746664509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc103900 msgr2=0x7f6ffc103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:52.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 --2- 172.21.15.3:0/3746664509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc103900 0x7f6ffc103d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009930 tx=0x7f6fe802a420 comp rx=0 tx=0).stop 2024-08-01T04:14:52.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 -- 172.21.15.3:0/3746664509 shutdown_connections 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 --2- 172.21.15.3:0/3746664509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ffc1042a0 0x7f6ffc1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 --2- 172.21.15.3:0/3746664509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc103900 0x7f6ffc103d60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 --2- 172.21.15.3:0/3746664509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ffc109900 0x7f6ffc109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 -- 172.21.15.3:0/3746664509 >> 172.21.15.3:0/3746664509 conn(0x7f6ffc0ff640 msgr2=0x7f6ffc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 -- 172.21.15.3:0/3746664509 shutdown_connections 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.300+0000 7f7002400640 1 -- 172.21.15.3:0/3746664509 wait complete. 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.301+0000 7f7002400640 1 Processor -- start 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.301+0000 7f7002400640 1 -- start start 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.301+0000 7f7002400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ffc103900 0x7f6ffc19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7002400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc1042a0 0x7f6ffc19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7002400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ffc109900 0x7f6ffc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7002400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113d20 con 0x7f6ffc103900 2024-08-01T04:14:52.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7002400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113ba0 con 0x7f6ffc109900 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7002400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113ea0 con 0x7f6ffc1042a0 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc1042a0 0x7f6ffc19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc1042a0 0x7f6ffc19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:57386/0 (socket says 172.21.15.3:57386) 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7001000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ffc103900 0x7f6ffc19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 -- 172.21.15.3:0/1801915894 learned_addr learned my addr 172.21.15.3:0/1801915894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f7001a00640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ffc109900 0x7f6ffc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 -- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ffc109900 msgr2=0x7f6ffc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ffc109900 0x7f6ffc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 -- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ffc103900 msgr2=0x7f6ffc19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ffc103900 0x7f6ffc19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:52.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.302+0000 7f6ffbe00640 1 -- 172.21.15.3:0/1801915894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc19f1b0 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.303+0000 7f6ffbe00640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc1042a0 0x7f6ffc19a740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6fe802a940 tx=0x7f6fe8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.303+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe803f070 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.303+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe8031c90 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.303+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8035030 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.303+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc19f440 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.303+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc19f8d0 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.304+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ffc062670 con 0x7f6ffc1042a0 2024-08-01T04:14:52.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.306+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6fe8042070 con 0x7f6ffc1042a0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.308+0000 7f6ff9600640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6fe40a2c10 0x7f6fe40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.308+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f6fe80efff0 con 0x7f6ffc1042a0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.308+0000 7f7001000640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6fe40a2c10 0x7f6fe40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.309+0000 7f6ff9600640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] conn(0x7f6fe40ac8c0 0x7f6fe40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.309+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 --> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fe8038e90 con 0x7f6fe40ac8c0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.309+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f6fe80f03a0 con 0x7f6ffc1042a0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.309+0000 7f7001a00640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] conn(0x7f6fe40ac8c0 0x7f6fe40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.309+0000 7f7001000640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6fe40a2c10 0x7f6fe40a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000bb60 tx=0x7f6ff000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.310+0000 7f7001a00640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] conn(0x7f6fe40ac8c0 0x7f6fe40aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.310+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== osd.10 v2:172.21.15.160:6824/1295991046 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6fe8038e90 con 0x7f6fe40ac8c0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.327+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 --> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ffc062af0 con 0x7f6fe40ac8c0 2024-08-01T04:14:52.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.327+0000 7f6ff9600640 1 -- 172.21.15.3:0/1801915894 <== osd.10 v2:172.21.15.160:6824/1295991046 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ffc062af0 con 0x7f6fe40ac8c0 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] conn(0x7f6fe40ac8c0 msgr2=0x7f6fe40aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] conn(0x7f6fe40ac8c0 0x7f6fe40aed60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6fe40a2c10 msgr2=0x7f6fe40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f6fe40a2c10 0x7f6fe40a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000bb60 tx=0x7f6ff000b390 comp rx=0 tx=0).stop 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc1042a0 msgr2=0x7f6ffc19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 --2- 172.21.15.3:0/1801915894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ffc1042a0 0x7f6ffc19a740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6fe802a940 tx=0x7f6fe8003b90 comp rx=0 tx=0).stop 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7001a00640 1 -- 172.21.15.3:0/1801915894 reap_dead start 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 shutdown_connections 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 >> 172.21.15.3:0/1801915894 conn(0x7f6ffc0ff640 msgr2=0x7f6ffc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 shutdown_connections 2024-08-01T04:14:52.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:52.328+0000 7f7002400640 1 -- 172.21.15.3:0/1801915894 wait complete. 2024-08-01T04:14:52.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:52 smithi163 ceph-mon[37383]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-08-01T04:14:53.226 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.226 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.226 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.226 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:53 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.229 INFO:teuthology.orchestra.run.smithi003.stdout:335007449112 2024-08-01T04:14:53.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-01T04:14:53.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:53 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:53 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:14:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 -- 172.21.15.3:0/3478768825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578101700 msgr2=0x7f0578101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 --2- 172.21.15.3:0/3478768825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578101700 0x7f0578101b60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0564009930 tx=0x7f056402a420 comp rx=0 tx=0).stop 2024-08-01T04:14:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 -- 172.21.15.3:0/3478768825 shutdown_connections 2024-08-01T04:14:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 --2- 172.21.15.3:0/3478768825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05781020a0 0x7f057810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 --2- 172.21.15.3:0/3478768825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578101700 0x7f0578101b60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 --2- 172.21.15.3:0/3478768825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578107700 0x7f0578107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.471+0000 7f057ec00640 1 -- 172.21.15.3:0/3478768825 >> 172.21.15.3:0/3478768825 conn(0x7f05780fd440 msgr2=0x7f05780ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.472+0000 7f057ec00640 1 -- 172.21.15.3:0/3478768825 shutdown_connections 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.472+0000 7f057ec00640 1 -- 172.21.15.3:0/3478768825 wait complete. 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.473+0000 7f057ec00640 1 Processor -- start 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.473+0000 7f057ec00640 1 -- start start 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ec00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578101700 0x7f0578198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ec00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05781020a0 0x7f0578198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578107700 0x7f057819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0578111b50 con 0x7f0578107700 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ec00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f05781119d0 con 0x7f05781020a0 2024-08-01T04:14:53.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ec00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0578111cd0 con 0x7f0578101700 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578101700 0x7f0578198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578101700 0x7f0578198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:57404/0 (socket says 172.21.15.3:57404) 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057d800640 1 -- 172.21.15.3:0/3413094976 learned_addr learned my addr 172.21.15.3:0/3413094976 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057ce00640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05781020a0 0x7f0578198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.474+0000 7f057e200640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578107700 0x7f057819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057d800640 1 -- 172.21.15.3:0/3413094976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05781020a0 msgr2=0x7f0578198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057d800640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05781020a0 0x7f0578198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057d800640 1 -- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578107700 msgr2=0x7f057819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057d800640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0578107700 0x7f057819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057d800640 1 -- 172.21.15.3:0/3413094976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057819cfe0 con 0x7f0578101700 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057d800640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578101700 0x7f0578198030 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f056c00e9f0 tx=0x7f056c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:53.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c00ccc0 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.475+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057819d2d0 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.476+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f056c00ce60 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.476+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c010430 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.476+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0578074150 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.477+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05780626b0 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.479+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f056c0105d0 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.480+0000 7f0576000640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f05600a2c10 0x7f05600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.480+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f056c0cf610 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.480+0000 7f057ce00640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f05600a2c10 0x7f05600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.481+0000 7f0576000640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] conn(0x7f05600ac8c0 0x7f05600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.481+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 --> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f056c012e50 con 0x7f05600ac8c0 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.481+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f056c0d6050 con 0x7f0578101700 2024-08-01T04:14:53.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.481+0000 7f057e200640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] conn(0x7f05600ac8c0 0x7f05600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.482+0000 7f057ce00640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f05600a2c10 0x7f05600a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0564006010 tx=0x7f0564000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.482+0000 7f057e200640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] conn(0x7f05600ac8c0 0x7f05600aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.483+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== osd.3 v2:172.21.15.3:6810/1408863124 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f056c012e50 con 0x7f05600ac8c0 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.500+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 --> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0578062b30 con 0x7f05600ac8c0 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.500+0000 7f0576000640 1 -- 172.21.15.3:0/3413094976 <== osd.3 v2:172.21.15.3:6810/1408863124 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0578062b30 con 0x7f05600ac8c0 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.500+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] conn(0x7f05600ac8c0 msgr2=0x7f05600aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] conn(0x7f05600ac8c0 0x7f05600aed60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f05600a2c10 msgr2=0x7f05600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f05600a2c10 0x7f05600a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0564006010 tx=0x7f0564000bc0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578101700 msgr2=0x7f0578198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 --2- 172.21.15.3:0/3413094976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0578101700 0x7f0578198030 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f056c00e9f0 tx=0x7f056c00edf0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057e200640 1 -- 172.21.15.3:0/3413094976 reap_dead start 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 shutdown_connections 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 >> 172.21.15.3:0/3413094976 conn(0x7f05780fd440 msgr2=0x7f05780fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 shutdown_connections 2024-08-01T04:14:53.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:53.501+0000 7f057ec00640 1 -- 172.21.15.3:0/3413094976 wait complete. 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 -- 172.21.15.3:0/418681802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c81031e0 msgr2=0x7fa0c810b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 --2- 172.21.15.3:0/418681802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c81031e0 0x7fa0c810b6d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009900 tx=0x7fa0b402a480 comp rx=0 tx=0).stop 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 -- 172.21.15.3:0/418681802 shutdown_connections 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 --2- 172.21.15.3:0/418681802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0c810bcf0 0x7fa0c810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 --2- 172.21.15.3:0/418681802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c81031e0 0x7fa0c810b6d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 --2- 172.21.15.3:0/418681802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 0x7fa0c8102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 -- 172.21.15.3:0/418681802 >> 172.21.15.3:0/418681802 conn(0x7fa0c80fc340 msgr2=0x7fa0c80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 -- 172.21.15.3:0/418681802 shutdown_connections 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.479+0000 7fa0cca00640 1 -- 172.21.15.3:0/418681802 wait complete. 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.480+0000 7fa0cca00640 1 Processor -- start 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.480+0000 7fa0cca00640 1 -- start start 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0cca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 0x7fa0c819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0cca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0c81031e0 0x7fa0c819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0cca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c810bcf0 0x7fa0c819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0cca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111b40 con 0x7fa0c810bcf0 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0cca00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81119c0 con 0x7fa0c81031e0 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0cca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8111cc0 con 0x7fa0c81028c0 2024-08-01T04:14:53.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 0x7fa0c819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 0x7fa0c819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54994/0 (socket says 172.21.15.3:54994) 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 -- 172.21.15.3:0/2452000672 learned_addr learned my addr 172.21.15.3:0/2452000672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c5e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0c81031e0 0x7fa0c819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c810bcf0 0x7fa0c819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 -- 172.21.15.3:0/2452000672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0c81031e0 msgr2=0x7fa0c819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0c81031e0 0x7fa0c819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 -- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c810bcf0 msgr2=0x7fa0c819ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0c810bcf0 0x7fa0c819ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.481+0000 7fa0c6800640 1 -- 172.21.15.3:0/2452000672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c819f1b0 con 0x7fa0c81028c0 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.482+0000 7fa0c6800640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 0x7fa0c819a200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800e9f0 tx=0x7fa0b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.482+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800cd80 con 0x7fa0c81028c0 2024-08-01T04:14:53.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.482+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b8005500 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.482+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8010690 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.482+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c819f4a0 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.482+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81a6d80 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.483+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0c80fe9b0 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.485+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa0b80046c0 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0bf400640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa0ac0a2c90 0x7fa0ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fa0b80cf5d0 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0c5e00640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa0ac0a2c90 0x7fa0ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0bf400640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] conn(0x7fa0ac0ac940 0x7fa0ac0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 --> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0ac0af4b0 con 0x7fa0ac0ac940 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa0b80d6050 con 0x7fa0c81028c0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.487+0000 7fa0c7200640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] conn(0x7fa0ac0ac940 0x7fa0ac0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:53.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.488+0000 7fa0c5e00640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa0ac0a2c90 0x7fa0ac0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.488+0000 7fa0c7200640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] conn(0x7fa0ac0ac940 0x7fa0ac0aede0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.489+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== osd.2 v2:172.21.15.163:6800/2416358231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0ac0af4b0 con 0x7fa0ac0ac940 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.505+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 --> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0c806a580 con 0x7fa0ac0ac940 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0bf400640 1 -- 172.21.15.3:0/2452000672 <== osd.2 v2:172.21.15.163:6800/2416358231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa0c806a580 con 0x7fa0ac0ac940 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] conn(0x7fa0ac0ac940 msgr2=0x7fa0ac0aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0cca00640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] conn(0x7fa0ac0ac940 0x7fa0ac0aede0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa0ac0a2c90 msgr2=0x7fa0ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0cca00640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa0ac0a2c90 0x7fa0ac0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b4005b50 comp rx=0 tx=0).stop 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 msgr2=0x7fa0c819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0cca00640 1 --2- 172.21.15.3:0/2452000672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0c81028c0 0x7fa0c819a200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800e9f0 tx=0x7fa0b800edf0 comp rx=0 tx=0).stop 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.506+0000 7fa0c7200640 1 -- 172.21.15.3:0/2452000672 reap_dead start 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.507+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 shutdown_connections 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.507+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 >> 172.21.15.3:0/2452000672 conn(0x7fa0c80fc340 msgr2=0x7fa0c80fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.507+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 shutdown_connections 2024-08-01T04:14:53.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:51.507+0000 7fa0cca00640 1 -- 172.21.15.3:0/2452000672 wait complete. 2024-08-01T04:14:54.274 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:54 smithi003 ceph-mon[32262]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:54.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:54 smithi160 ceph-mon[37224]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:54.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:54 smithi163 ceph-mon[37383]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:54.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.538+0000 7f3436400640 1 -- 172.21.15.3:0/4112530202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3430107d50 msgr2=0x7f3430108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:54.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.538+0000 7f3436400640 1 --2- 172.21.15.3:0/4112530202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3430107d50 0x7f3430108170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3420009930 tx=0x7f342002a420 comp rx=0 tx=0).stop 2024-08-01T04:14:54.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 -- 172.21.15.3:0/4112530202 shutdown_connections 2024-08-01T04:14:54.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 --2- 172.21.15.3:0/4112530202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34301086b0 0x7f3430114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 --2- 172.21.15.3:0/4112530202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3430107d50 0x7f3430108170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 --2- 172.21.15.3:0/4112530202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f343010dd90 0x7f343010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 -- 172.21.15.3:0/4112530202 >> 172.21.15.3:0/4112530202 conn(0x7f34300776b0 msgr2=0x7f3430079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 -- 172.21.15.3:0/4112530202 shutdown_connections 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.539+0000 7f3436400640 1 -- 172.21.15.3:0/4112530202 wait complete. 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.540+0000 7f3436400640 1 Processor -- start 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.540+0000 7f3436400640 1 -- start start 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.540+0000 7f3436400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107d50 0x7f343019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3436400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34301086b0 0x7f343019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3436400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343010dd90 0x7f34301a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:54.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3436400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34301180b0 con 0x7f343010dd90 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3436400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3430117f30 con 0x7f3430107d50 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3436400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3430118230 con 0x7f34301086b0 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343010dd90 0x7f34301a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107d50 0x7f343019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f342fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34301086b0 0x7f343019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107d50 0x7f343019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:37066/0 (socket says 172.21.15.3:37066) 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343010dd90 0x7f34301a2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39320/0 (socket says 172.21.15.3:39320) 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 -- 172.21.15.3:0/15798056 learned_addr learned my addr 172.21.15.3:0/15798056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 -- 172.21.15.3:0/15798056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34301086b0 msgr2=0x7f343019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34301086b0 0x7f343019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 -- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343010dd90 msgr2=0x7f34301a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:54.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343010dd90 0x7f34301a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.541+0000 7f3435000640 1 -- 172.21.15.3:0/15798056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34301a35d0 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.542+0000 7f3435000640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107d50 0x7f343019e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f342400e9f0 tx=0x7f342400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.542+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400cd80 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.542+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34301a38c0 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.542+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3424005500 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.542+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34301ab1a0 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.542+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424010690 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.543+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34300626b0 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.545+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f34240046c0 con 0x7f3430107d50 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.547+0000 7f342d600640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f34180a2c90 0x7f34180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:54.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.547+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f34240cf920 con 0x7f3430107d50 2024-08-01T04:14:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.547+0000 7f342fe00640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f34180a2c90 0x7f34180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.547+0000 7f342d600640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] conn(0x7f34180ac940 0x7f34180aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.547+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 --> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3424004470 con 0x7f34180ac940 2024-08-01T04:14:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.547+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f34240cfcd0 con 0x7f3430107d50 2024-08-01T04:14:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.548+0000 7f3435a00640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] conn(0x7f34180ac940 0x7f34180aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:54.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.548+0000 7f342fe00640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f34180a2c90 0x7f34180a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3420009900 tx=0x7f34200058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.548+0000 7f3435a00640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] conn(0x7f34180ac940 0x7f34180aede0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.549+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== osd.7 v2:172.21.15.160:6816/3141457315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3424004470 con 0x7f34180ac940 2024-08-01T04:14:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.564+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 --> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3430062ab0 con 0x7f34180ac940 2024-08-01T04:14:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.565+0000 7f342d600640 1 -- 172.21.15.3:0/15798056 <== osd.7 v2:172.21.15.160:6816/3141457315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3430062ab0 con 0x7f34180ac940 2024-08-01T04:14:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] conn(0x7f34180ac940 msgr2=0x7f34180aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:54.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] conn(0x7f34180ac940 0x7f34180aede0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f34180a2c90 msgr2=0x7f34180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f34180a2c90 0x7f34180a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3420009900 tx=0x7f34200058d0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107d50 msgr2=0x7f343019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 --2- 172.21.15.3:0/15798056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3430107d50 0x7f343019e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f342400e9f0 tx=0x7f342400edf0 comp rx=0 tx=0).stop 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3435a00640 1 -- 172.21.15.3:0/15798056 reap_dead start 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 shutdown_connections 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 >> 172.21.15.3:0/15798056 conn(0x7f34300776b0 msgr2=0x7f3430079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 shutdown_connections 2024-08-01T04:14:54.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:54.566+0000 7f3436400640 1 -- 172.21.15.3:0/15798056 wait complete. 2024-08-01T04:14:55.224 INFO:teuthology.orchestra.run.smithi003.stdout:283467841570 2024-08-01T04:14:55.224 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-01T04:14:55.987 INFO:teuthology.orchestra.run.smithi003.stdout:107374182465 2024-08-01T04:14:55.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-01T04:14:56.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 -- 172.21.15.3:0/2675588138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c1078b0 msgr2=0x7fa90c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 --2- 172.21.15.3:0/2675588138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c1078b0 0x7fa90c107c90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009930 tx=0x7fa8fc02a420 comp rx=0 tx=0).stop 2024-08-01T04:14:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 -- 172.21.15.3:0/2675588138 shutdown_connections 2024-08-01T04:14:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 --2- 172.21.15.3:0/2675588138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa90c102250 0x7fa90c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 --2- 172.21.15.3:0/2675588138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa90c1018b0 0x7fa90c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 --2- 172.21.15.3:0/2675588138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c1078b0 0x7fa90c107c90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 -- 172.21.15.3:0/2675588138 >> 172.21.15.3:0/2675588138 conn(0x7fa90c0fd5f0 msgr2=0x7fa90c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:56.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 -- 172.21.15.3:0/2675588138 shutdown_connections 2024-08-01T04:14:56.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.303+0000 7fa911a00640 1 -- 172.21.15.3:0/2675588138 wait complete. 2024-08-01T04:14:56.324 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:56 smithi003 ceph-mon[32262]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.304+0000 7fa911a00640 1 Processor -- start 2024-08-01T04:14:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.304+0000 7fa911a00640 1 -- start start 2024-08-01T04:14:56.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.304+0000 7fa911a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa90c1018b0 0x7fa90c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa911a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c102250 0x7fa90c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa911a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa90c1078b0 0x7fa90c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa911a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c111d00 con 0x7fa90c102250 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa911a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c111b80 con 0x7fa90c1078b0 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa911a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c111e80 con 0x7fa90c1018b0 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c102250 0x7fa90c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa90c1018b0 0x7fa90c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c102250 0x7fa90c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39342/0 (socket says 172.21.15.3:39342) 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90a000640 1 -- 172.21.15.3:0/1734915659 learned_addr learned my addr 172.21.15.3:0/1734915659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90b400640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa90c1078b0 0x7fa90c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90aa00640 1 -- 172.21.15.3:0/1734915659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa90c1078b0 msgr2=0x7fa90c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90aa00640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa90c1078b0 0x7fa90c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90aa00640 1 -- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c102250 msgr2=0x7fa90c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90aa00640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c102250 0x7fa90c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.305+0000 7fa90aa00640 1 -- 172.21.15.3:0/1734915659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c19f370 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.306+0000 7fa90a000640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90c102250 0x7fa90c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.306+0000 7fa90aa00640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa90c1018b0 0x7fa90c19a3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc02a940 tx=0x7fa8fc003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.306+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc03f070 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.306+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc031c90 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.306+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc035030 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.306+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c19f600 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.307+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c19fa90 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.308+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa90c0feec0 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.309+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa8fc042070 con 0x7fa90c1018b0 2024-08-01T04:14:56.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.311+0000 7fa903400640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa8f00a2c90 0x7fa8f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.311+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fa8fc0efff0 con 0x7fa90c1018b0 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.311+0000 7fa90a000640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa8f00a2c90 0x7fa8f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.311+0000 7fa903400640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] conn(0x7fa8f00ac940 0x7fa8f00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.311+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 --> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8fc038e90 con 0x7fa8f00ac940 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.311+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa8fc0f0400 con 0x7fa90c1018b0 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.312+0000 7fa90b400640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] conn(0x7fa8f00ac940 0x7fa8f00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.312+0000 7fa90a000640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa8f00a2c90 0x7fa8f00a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80095f0 tx=0x7fa8f8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.313+0000 7fa90b400640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] conn(0x7fa8f00ac940 0x7fa8f00aede0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:56.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.314+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== osd.5 v2:172.21.15.163:6808/1207930650 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8fc038e90 con 0x7fa8f00ac940 2024-08-01T04:14:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.336+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 --> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa90c102ff0 con 0x7fa8f00ac940 2024-08-01T04:14:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.337+0000 7fa903400640 1 -- 172.21.15.3:0/1734915659 <== osd.5 v2:172.21.15.163:6808/1207930650 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa90c102ff0 con 0x7fa8f00ac940 2024-08-01T04:14:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] conn(0x7fa8f00ac940 msgr2=0x7fa8f00aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] conn(0x7fa8f00ac940 0x7fa8f00aede0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa8f00a2c90 msgr2=0x7fa8f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa8f00a2c90 0x7fa8f00a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80095f0 tx=0x7fa8f8005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa90c1018b0 msgr2=0x7fa90c19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 --2- 172.21.15.3:0/1734915659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa90c1018b0 0x7fa90c19a3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc02a940 tx=0x7fa8fc003b20 comp rx=0 tx=0).stop 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa90b400640 1 -- 172.21.15.3:0/1734915659 reap_dead start 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 shutdown_connections 2024-08-01T04:14:56.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.338+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 >> 172.21.15.3:0/1734915659 conn(0x7fa90c0fd5f0 msgr2=0x7fa90c10cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:56.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.339+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 shutdown_connections 2024-08-01T04:14:56.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:56.339+0000 7fa911a00640 1 -- 172.21.15.3:0/1734915659 wait complete. 2024-08-01T04:14:56.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:56 smithi160 ceph-mon[37224]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:56.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:56 smithi163 ceph-mon[37383]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 -- 172.21.15.3:0/2141775366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 msgr2=0x7f0ef0112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 --2- 172.21.15.3:0/2141775366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef0112220 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009900 tx=0x7f0ed802a410 comp rx=0 tx=0).stop 2024-08-01T04:14:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 -- 172.21.15.3:0/2141775366 shutdown_connections 2024-08-01T04:14:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 --2- 172.21.15.3:0/2141775366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ef0112760 0x7f0ef0114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 --2- 172.21.15.3:0/2141775366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef0112220 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 --2- 172.21.15.3:0/2141775366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ef0076450 0x7f0ef0076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 -- 172.21.15.3:0/2141775366 >> 172.21.15.3:0/2141775366 conn(0x7f0ef00ff7a0 msgr2=0x7f0ef0101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 -- 172.21.15.3:0/2141775366 shutdown_connections 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.742+0000 7f0ef5200640 1 -- 172.21.15.3:0/2141775366 wait complete. 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.743+0000 7f0ef5200640 1 Processor -- start 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.743+0000 7f0ef5200640 1 -- start start 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0ef5200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ef0076450 0x7f0ef01a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0ef5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef01a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0ef5200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ef0112760 0x7f0ef019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0ef5200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef0118330 con 0x7f0ef0076d70 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0ef5200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01181b0 con 0x7f0ef0076450 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0ef5200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef01184b0 con 0x7f0ef0112760 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef01a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef01a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39336/0 (socket says 172.21.15.3:39336) 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 -- 172.21.15.3:0/4117457812 learned_addr learned my addr 172.21.15.3:0/4117457812 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eede00640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ef0076450 0x7f0ef01a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eee800640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ef0112760 0x7f0ef019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 -- 172.21.15.3:0/4117457812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ef0112760 msgr2=0x7f0ef019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ef0112760 0x7f0ef019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 -- 172.21.15.3:0/4117457812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ef0076450 msgr2=0x7f0ef01a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.744+0000 7f0eed400640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ef0076450 0x7f0ef01a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0eed400640 1 -- 172.21.15.3:0/4117457812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019ed30 con 0x7f0ef0076d70 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0eed400640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef01a4980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8004770 tx=0x7f0ed8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed803f070 con 0x7f0ef0076d70 2024-08-01T04:14:56.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ed8031ca0 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef019efc0 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8035030 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.745+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01ab310 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.746+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ef00626b0 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.748+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0ed8042070 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.750+0000 7f0ee6a00640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0ed40a2c90 0x7f0ed40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.750+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f0ed80efdc0 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.750+0000 7f0eede00640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0ed40a2c90 0x7f0ed40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.750+0000 7f0ee6a00640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] conn(0x7f0ed40ac810 0x7f0ed40aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.750+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 --> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ed8032e50 con 0x7f0ed40ac810 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.750+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0ed80f0170 con 0x7f0ef0076d70 2024-08-01T04:14:56.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.751+0000 7f0eee800640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] conn(0x7f0ed40ac810 0x7f0ed40aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.751+0000 7f0eede00640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0ed40a2c90 0x7f0ed40a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00095f0 tx=0x7f0ee0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.751+0000 7f0eee800640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] conn(0x7f0ed40ac810 0x7f0ed40aecb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.752+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== osd.6 v2:172.21.15.3:6818/885876521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ed8032e50 con 0x7f0ed40ac810 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.767+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 --> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ef0062ab0 con 0x7f0ed40ac810 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.768+0000 7f0ee6a00640 1 -- 172.21.15.3:0/4117457812 <== osd.6 v2:172.21.15.3:6818/885876521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0ef0062ab0 con 0x7f0ed40ac810 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.768+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] conn(0x7f0ed40ac810 msgr2=0x7f0ed40aecb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.768+0000 7f0ef5200640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] conn(0x7f0ed40ac810 0x7f0ed40aecb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:14:56.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.768+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0ed40a2c90 msgr2=0x7f0ed40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f0ed40a2c90 0x7f0ed40a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0ee00095f0 tx=0x7f0ee0005f90 comp rx=0 tx=0).stop 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 msgr2=0x7f0ef01a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 --2- 172.21.15.3:0/4117457812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef0076d70 0x7f0ef01a4980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8004770 tx=0x7f0ed8003c10 comp rx=0 tx=0).stop 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0eee800640 1 -- 172.21.15.3:0/4117457812 reap_dead start 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 shutdown_connections 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 >> 172.21.15.3:0/4117457812 conn(0x7f0ef00ff7a0 msgr2=0x7f0ef0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 shutdown_connections 2024-08-01T04:14:56.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:14:55.769+0000 7f0ef5200640 1 -- 172.21.15.3:0/4117457812 wait complete. 2024-08-01T04:14:57.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:57 smithi003 ceph-mon[32262]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:57.326 INFO:teuthology.orchestra.run.smithi003.stdout:197568495667 2024-08-01T04:14:57.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-01T04:14:57.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:57 smithi160 ceph-mon[37224]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:57.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:57 smithi163 ceph-mon[37383]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:57.840 INFO:teuthology.orchestra.run.smithi003.stdout:77309411401 2024-08-01T04:14:57.840 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-01T04:14:58.059 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:58.221 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:14:59.053 INFO:teuthology.orchestra.run.smithi003.stdout:163208757305 2024-08-01T04:14:59.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-01T04:14:59.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:14:59 smithi003 ceph-mon[32262]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:59.414 INFO:teuthology.orchestra.run.smithi003.stdout:304942678047 2024-08-01T04:14:59.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-01T04:14:59.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:14:59 smithi163 ceph-mon[37383]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:59.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:14:59 smithi160 ceph-mon[37224]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:14:59.763 INFO:teuthology.orchestra.run.smithi003.stdout:141733920829 2024-08-01T04:14:59.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-01T04:15:00.767 INFO:teuthology.orchestra.run.smithi003.stdout:219043332142 2024-08-01T04:15:00.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-01T04:15:00.844 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 -- 172.21.15.3:0/2612106008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e4074240 msgr2=0x7f73e4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2612106008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e4074240 0x7f73e4074620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009900 tx=0x7f73d802a480 comp rx=0 tx=0).stop 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 -- 172.21.15.3:0/2612106008 shutdown_connections 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2612106008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e410e180 0x7f73e4110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2612106008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074b60 0x7f73e410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2612106008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e4074240 0x7f73e4074620 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.114+0000 7f73ebe00640 1 -- 172.21.15.3:0/2612106008 >> 172.21.15.3:0/2612106008 conn(0x7f73e40fd5e0 msgr2=0x7f73e40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:01.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.116+0000 7f73ebe00640 1 -- 172.21.15.3:0/2612106008 shutdown_connections 2024-08-01T04:15:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.116+0000 7f73ebe00640 1 -- 172.21.15.3:0/2612106008 wait complete. 2024-08-01T04:15:01.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.117+0000 7f73ebe00640 1 Processor -- start 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 -- start start 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 0x7f73e419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e4074b60 0x7f73e419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e410e180 0x7f73e419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4113f00 con 0x7f73e410e180 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4113d80 con 0x7f73e4074b60 2024-08-01T04:15:01.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.118+0000 7f73ebe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4114080 con 0x7f73e4074240 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 0x7f73e419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 0x7f73e419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:57504/0 (socket says 172.21.15.3:57504) 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 -- 172.21.15.3:0/2681756346 learned_addr learned my addr 172.21.15.3:0/2681756346 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e8a00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e4074b60 0x7f73e419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9e00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e410e180 0x7f73e419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 -- 172.21.15.3:0/2681756346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e4074b60 msgr2=0x7f73e419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e4074b60 0x7f73e419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 -- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e410e180 msgr2=0x7f73e419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e410e180 0x7f73e419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.119+0000 7f73e9400640 1 -- 172.21.15.3:0/2681756346 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e419f350 con 0x7f73e4074240 2024-08-01T04:15:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.120+0000 7f73e9400640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 0x7f73e419a3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73d8004770 tx=0x7f73d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.120+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d803f070 con 0x7f73e4074240 2024-08-01T04:15:01.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.120+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e419f5e0 con 0x7f73e4074240 2024-08-01T04:15:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.121+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e41a6f20 con 0x7f73e4074240 2024-08-01T04:15:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.121+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8031ca0 con 0x7f73e4074240 2024-08-01T04:15:01.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.121+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8035030 con 0x7f73e4074240 2024-08-01T04:15:01.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.123+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c4005290 con 0x7f73e4074240 2024-08-01T04:15:01.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.127+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f73d8042070 con 0x7f73e4074240 2024-08-01T04:15:01.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.128+0000 7f73e2000640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.128+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f73d80ef710 con 0x7f73e4074240 2024-08-01T04:15:01.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.128+0000 7f73e8a00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.130+0000 7f73e8a00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73e40ffb70 tx=0x7f73d000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:01.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.132+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f73d8049030 con 0x7f73e4074240 2024-08-01T04:15:01.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.356+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f73c4005850 con 0x7f73e4074240 2024-08-01T04:15:01.359 INFO:teuthology.orchestra.run.smithi003.stdout:55834574926 2024-08-01T04:15:01.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.357+0000 7f73e2000640 1 -- 172.21.15.3:0/2681756346 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73d80bc490 con 0x7f73e4074240 2024-08-01T04:15:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f73cc0a2c10 msgr2=0x7f73cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f73e40ffb70 tx=0x7f73d000b390 comp rx=0 tx=0).stop 2024-08-01T04:15:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 msgr2=0x7f73e419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 0x7f73e419a3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73d8004770 tx=0x7f73d8003c10 comp rx=0 tx=0).stop 2024-08-01T04:15:01.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 shutdown_connections 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f73cc0a2c10 0x7f73cc0a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73e410e180 0x7f73e419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e4074b60 0x7f73e419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 --2- 172.21.15.3:0/2681756346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73e4074240 0x7f73e419a3a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.361+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 >> 172.21.15.3:0/2681756346 conn(0x7f73e40fd5e0 msgr2=0x7f73e40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.362+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 shutdown_connections 2024-08-01T04:15:01.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.362+0000 7f73ebe00640 1 -- 172.21.15.3:0/2681756346 wait complete. 2024-08-01T04:15:01.380 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:01 smithi003 ceph-mon[32262]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:01.381 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:01 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3751395687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-01T04:15:01.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:01 smithi163 ceph-mon[37383]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:01.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:01 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3751395687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-01T04:15:01.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:01 smithi160 ceph-mon[37224]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:01.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:01 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3751395687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.994+0000 7f7c43e00640 1 -- 172.21.15.3:0/3458113350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c441042a0 msgr2=0x7f7c441107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.994+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3458113350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c441042a0 0x7f7c441107d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009cd0 tx=0x7f7c3c02a8e0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.994+0000 7f7c43e00640 1 -- 172.21.15.3:0/3458113350 shutdown_connections 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.994+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3458113350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c441042a0 0x7f7c441107d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.994+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3458113350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c44103900 0x7f7c44103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.994+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3458113350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44109900 0x7f7c44109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.995+0000 7f7c43e00640 1 -- 172.21.15.3:0/3458113350 >> 172.21.15.3:0/3458113350 conn(0x7f7c440ff640 msgr2=0x7f7c44101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:01.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.997+0000 7f7c43e00640 1 -- 172.21.15.3:0/3458113350 shutdown_connections 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.998+0000 7f7c43e00640 1 -- 172.21.15.3:0/3458113350 wait complete. 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 Processor -- start 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 -- start start 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 0x7f7c4419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c441042a0 0x7f7c4419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c44109900 0x7f7c4419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44113d50 con 0x7f7c44109900 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44113bd0 con 0x7f7c441042a0 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:00.999+0000 7f7c43e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44113ed0 con 0x7f7c44103900 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 0x7f7c4419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 0x7f7c4419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:57484/0 (socket says 172.21.15.3:57484) 2024-08-01T04:15:01.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 -- 172.21.15.3:0/3751395687 learned_addr learned my addr 172.21.15.3:0/3751395687 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42000640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c441042a0 0x7f7c4419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c43400640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c44109900 0x7f7c4419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 -- 172.21.15.3:0/3751395687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c441042a0 msgr2=0x7f7c4419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c441042a0 0x7f7c4419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 -- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c44109900 msgr2=0x7f7c4419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c44109900 0x7f7c4419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42a00640 1 -- 172.21.15.3:0/3751395687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4419f180 con 0x7f7c44103900 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.000+0000 7f7c42000640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c441042a0 0x7f7c4419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.001+0000 7f7c42a00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 0x7f7c4419a1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9f0 tx=0x7f7c3400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.001+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400cd80 con 0x7f7c44103900 2024-08-01T04:15:01.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.001+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4419f470 con 0x7f7c44103900 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.001+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c34005500 con 0x7f7c44103900 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:249108103211 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.001+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010690 con 0x7f7c44103900 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.001+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441a6d50 con 0x7f7c44103900 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.003+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44105040 con 0x7f7c44103900 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.004+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7c340046c0 con 0x7f7c44103900 2024-08-01T04:15:01.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.006+0000 7f7c3b400640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7c280a2c10 0x7f7c280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.006+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f7c340cf5f0 con 0x7f7c44103900 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.006+0000 7f7c42000640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7c280a2c10 0x7f7c280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.007+0000 7f7c42000640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7c280a2c10 0x7f7c280a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7c300095f0 tx=0x7f7c30005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.010+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7c3409c370 con 0x7f7c44103900 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.222+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7c44103d60 con 0x7f7c44103900 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.223+0000 7f7c3b400640 1 -- 172.21.15.3:0/3751395687 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c3409bc70 con 0x7f7c44103900 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7c280a2c10 msgr2=0x7f7c280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7c280a2c10 0x7f7c280a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7c300095f0 tx=0x7f7c30005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 msgr2=0x7f7c4419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 0x7f7c4419a1d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9f0 tx=0x7f7c3400edf0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 shutdown_connections 2024-08-01T04:15:01.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7c280a2c10 0x7f7c280a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c44109900 0x7f7c4419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c441042a0 0x7f7c4419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 --2- 172.21.15.3:0/3751395687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c44103900 0x7f7c4419a1d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 >> 172.21.15.3:0/3751395687 conn(0x7f7c440ff640 msgr2=0x7f7c440ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 shutdown_connections 2024-08-01T04:15:01.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.226+0000 7f7c43e00640 1 -- 172.21.15.3:0/3751395687 wait complete. 2024-08-01T04:15:02.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 -- 172.21.15.3:0/130322318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 msgr2=0x7f66181107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:02.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 --2- 172.21.15.3:0/130322318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f66181107d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6610009cd0 tx=0x7f661002a8f0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 -- 172.21.15.3:0/130322318 shutdown_connections 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 --2- 172.21.15.3:0/130322318 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f66181107d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 --2- 172.21.15.3:0/130322318 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6618103900 0x7f6618103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 --2- 172.21.15.3:0/130322318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618109900 0x7f6618109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 -- 172.21.15.3:0/130322318 >> 172.21.15.3:0/130322318 conn(0x7f66180ff600 msgr2=0x7f6618101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 -- 172.21.15.3:0/130322318 shutdown_connections 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.901+0000 7f661ee00640 1 -- 172.21.15.3:0/130322318 wait complete. 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.902+0000 7f661ee00640 1 Processor -- start 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.902+0000 7f661ee00640 1 -- start start 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.902+0000 7f661ee00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618103900 0x7f661819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661ee00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f661819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661ee00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6618109900 0x7f661819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:02.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661ee00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6618113d00 con 0x7f66181042a0 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661ee00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6618113b80 con 0x7f6618109900 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661ee00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6618113e80 con 0x7f6618103900 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f661819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f661819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51786/0 (socket says 172.21.15.3:51786) 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661d000640 1 -- 172.21.15.3:0/2703090054 learned_addr learned my addr 172.21.15.3:0/2703090054 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661da00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618103900 0x7f661819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661e400640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6618109900 0x7f661819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661da00640 1 -- 172.21.15.3:0/2703090054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6618109900 msgr2=0x7f661819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661da00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6618109900 0x7f661819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661da00640 1 -- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 msgr2=0x7f661819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:02.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661da00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f661819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661da00640 1 -- 172.21.15.3:0/2703090054 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661819f150 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.903+0000 7f661d000640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f661819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.904+0000 7f661da00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618103900 0x7f661819a1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f660c00e9f0 tx=0x7f660c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.904+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c00ccc0 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.904+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f660c010910 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.904+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c018960 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.904+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661819f440 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.904+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66181a6d20 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.905+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6618103d60 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.907+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f660c010ac0 con 0x7f6618103900 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.909+0000 7f6616000640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f66000a2c10 0x7f66000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:02.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.909+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f660c0cf570 con 0x7f6618103900 2024-08-01T04:15:02.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.909+0000 7f661d000640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f66000a2c10 0x7f66000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:02.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.910+0000 7f661d000640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f66000a2c10 0x7f66000a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6618101bf0 tx=0x7f6604005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:02.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:01.913+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f660c09bca0 con 0x7f6618103900 2024-08-01T04:15:02.511 INFO:teuthology.orchestra.run.smithi003.stdout:283467841572 2024-08-01T04:15:02.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.108+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f661810b8e0 con 0x7f6618103900 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.108+0000 7f6616000640 1 -- 172.21.15.3:0/2703090054 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f660c09ffa0 con 0x7f6618103900 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.111+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f66000a2c10 msgr2=0x7f66000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.111+0000 7f661ee00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f66000a2c10 0x7f66000a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6618101bf0 tx=0x7f6604005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.111+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618103900 msgr2=0x7f661819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.111+0000 7f661ee00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618103900 0x7f661819a1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f660c00e9f0 tx=0x7f660c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 shutdown_connections 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f66000a2c10 0x7f66000a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6618109900 0x7f661819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66181042a0 0x7f661819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 --2- 172.21.15.3:0/2703090054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6618103900 0x7f661819a1a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 >> 172.21.15.3:0/2703090054 conn(0x7f66180ff600 msgr2=0x7f66180ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:02.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 shutdown_connections 2024-08-01T04:15:02.513 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:02 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2681756346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-01T04:15:02.513 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:02 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2703090054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-01T04:15:02.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:02.112+0000 7f661ee00640 1 -- 172.21.15.3:0/2703090054 wait complete. 2024-08-01T04:15:02.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:02 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2681756346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-01T04:15:02.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:02 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2703090054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-01T04:15:02.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:02 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2681756346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-01T04:15:02.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:02 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2703090054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-01T04:15:03.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574926 for osd.1 2024-08-01T04:15:03.440 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:03.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:03 smithi163 ceph-mon[37383]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:03.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:03 smithi003 ceph-mon[32262]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:03.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:03 smithi160 ceph-mon[37224]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:03.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841570 got 283467841572 for osd.9 2024-08-01T04:15:03.881 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.062 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103209 got 249108103211 for osd.8 2024-08-01T04:15:05.062 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.344 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.348 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.349 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:05.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:05 smithi163 ceph-mon[37383]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:05.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:05 smithi003 ceph-mon[32262]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:05.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:05 smithi160 ceph-mon[37224]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 -- 172.21.15.3:0/2501581422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0074350 msgr2=0x7feaf0074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 --2- 172.21.15.3:0/2501581422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0074350 0x7feaf0074730 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fead8009900 tx=0x7fead802a480 comp rx=0 tx=0).stop 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 -- 172.21.15.3:0/2501581422 shutdown_connections 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 --2- 172.21.15.3:0/2501581422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf010e0f0 0x7feaf0110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 --2- 172.21.15.3:0/2501581422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 --2- 172.21.15.3:0/2501581422 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf0074350 0x7feaf0074730 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 -- 172.21.15.3:0/2501581422 >> 172.21.15.3:0/2501581422 conn(0x7feaf00fd5f0 msgr2=0x7feaf00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 -- 172.21.15.3:0/2501581422 shutdown_connections 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.970+0000 7feaf5600640 1 -- 172.21.15.3:0/2501581422 wait complete. 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.971+0000 7feaf5600640 1 Processor -- start 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 -- start start 2024-08-01T04:15:06.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf0074350 0x7feaf019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf010e0f0 0x7feaf019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0113ea0 con 0x7feaf010e0f0 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0113d20 con 0x7feaf0074350 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaf5600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0114020 con 0x7feaf0074c70 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaef400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf010e0f0 0x7feaf019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaee000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaeea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf0074350 0x7feaf019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaee000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:46726/0 (socket says 172.21.15.3:46726) 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.972+0000 7feaee000640 1 -- 172.21.15.3:0/3889879219 learned_addr learned my addr 172.21.15.3:0/3889879219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaee000640 1 -- 172.21.15.3:0/3889879219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf0074350 msgr2=0x7feaf019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaee000640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf0074350 0x7feaf019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaee000640 1 -- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf010e0f0 msgr2=0x7feaf019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaee000640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf010e0f0 0x7feaf019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaee000640 1 -- 172.21.15.3:0/3889879219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf019f370 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaee000640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf019a900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feae000eb90 tx=0x7feae000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0004b20 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.973+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf019f660 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.974+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0004cc0 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.974+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0010720 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.974+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf01a6f40 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.975+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf0074730 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.977+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7feae0005020 con 0x7feaf0074c70 2024-08-01T04:15:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.978+0000 7feae7400640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fead40a2c90 0x7fead40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.978+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7feae00cf800 con 0x7feaf0074c70 2024-08-01T04:15:06.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.979+0000 7feaeea00640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fead40a2c90 0x7fead40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.980+0000 7feaeea00640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fead40a2c90 0x7fead40a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fead8009230 tx=0x7fead80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:06.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:05.983+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feae009c580 con 0x7feaf0074c70 2024-08-01T04:15:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.080+0000 7f3f1f200640 1 -- 172.21.15.3:0/3004539694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 msgr2=0x7f3f18074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 --2- 172.21.15.3:0/3004539694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f18074730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009900 tx=0x7f3f0c02a480 comp rx=0 tx=0).stop 2024-08-01T04:15:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 -- 172.21.15.3:0/3004539694 shutdown_connections 2024-08-01T04:15:06.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 --2- 172.21.15.3:0/3004539694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f1810df40 0x7f3f18110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 --2- 172.21.15.3:0/3004539694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f18074c70 0x7f3f1810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 --2- 172.21.15.3:0/3004539694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f18074730 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 -- 172.21.15.3:0/3004539694 >> 172.21.15.3:0/3004539694 conn(0x7f3f180fd440 msgr2=0x7f3f180ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 -- 172.21.15.3:0/3004539694 shutdown_connections 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.081+0000 7f3f1f200640 1 -- 172.21.15.3:0/3004539694 wait complete. 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 Processor -- start 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 -- start start 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f1819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f18074c70 0x7f3f1819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f1810df40 0x7f3f1819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18113d40 con 0x7f3f18074c70 2024-08-01T04:15:06.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18113bc0 con 0x7f3f1810df40 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.082+0000 7f3f1f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18113ec0 con 0x7f3f18074350 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f1819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f18074c70 0x7f3f1819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f1819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:46748/0 (socket says 172.21.15.3:46748) 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 -- 172.21.15.3:0/2193495415 learned_addr learned my addr 172.21.15.3:0/2193495415 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1e800640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f1810df40 0x7f3f1819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 -- 172.21.15.3:0/2193495415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f1810df40 msgr2=0x7f3f1819ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f1810df40 0x7f3f1819ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 -- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f18074c70 msgr2=0x7f3f1819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f18074c70 0x7f3f1819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1de00640 1 -- 172.21.15.3:0/2193495415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1819f170 con 0x7f3f18074350 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.083+0000 7f3f1d400640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f18074c70 0x7f3f1819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:06.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.084+0000 7f3f1de00640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f1819a1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c004770 tx=0x7f3f0c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.084+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c03f070 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.084+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f0c031ca0 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.084+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c035030 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.084+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1819f400 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.084+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f181a6d40 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.085+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f18074730 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.087+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3f0c042070 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.089+0000 7f3f12a00640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f3f000a2c90 0x7f3f000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.089+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f3f0c0ef9d0 con 0x7f3f18074350 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.089+0000 7f3f1d400640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f3f000a2c90 0x7f3f000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.090+0000 7f3f1d400640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f3f000a2c90 0x7f3f000a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f1810f330 tx=0x7f3f08005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:06.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.090+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3f0c0bc1c0 con 0x7f3f18074350 2024-08-01T04:15:06.186 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:06.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.190+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7feaf019b720 con 0x7feaf0074c70 2024-08-01T04:15:06.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.191+0000 7feae7400640 1 -- 172.21.15.3:0/3889879219 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feaf019b720 con 0x7feaf0074c70 2024-08-01T04:15:06.193 INFO:teuthology.orchestra.run.smithi003.stdout:107374182468 2024-08-01T04:15:06.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fead40a2c90 msgr2=0x7fead40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fead40a2c90 0x7fead40a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fead8009230 tx=0x7fead80058d0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 msgr2=0x7feaf019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf019a900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feae000eb90 tx=0x7feae000ef90 comp rx=0 tx=0).stop 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 shutdown_connections 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fead40a2c90 0x7fead40a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf010e0f0 0x7feaf019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0074c70 0x7feaf019a900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 --2- 172.21.15.3:0/3889879219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf0074350 0x7feaf019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.195+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 >> 172.21.15.3:0/3889879219 conn(0x7feaf00fd5f0 msgr2=0x7feaf00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.196+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 shutdown_connections 2024-08-01T04:15:06.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.196+0000 7feaf5600640 1 -- 172.21.15.3:0/3889879219 wait complete. 2024-08-01T04:15:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.313+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3f181090f0 con 0x7f3f18074350 2024-08-01T04:15:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.314+0000 7f3f12a00640 1 -- 172.21.15.3:0/2193495415 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f0c0c04c0 con 0x7f3f18074350 2024-08-01T04:15:06.316 INFO:teuthology.orchestra.run.smithi003.stdout:34359738451 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.316+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f3f000a2c90 msgr2=0x7f3f000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.316+0000 7f3f1f200640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f3f000a2c90 0x7f3f000a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f1810f330 tx=0x7f3f08005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.316+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 msgr2=0x7f3f1819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.316+0000 7f3f1f200640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f1819a1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c004770 tx=0x7f3f0c003c10 comp rx=0 tx=0).stop 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 shutdown_connections 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f3f000a2c90 0x7f3f000a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f1810df40 0x7f3f1819ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3f18074c70 0x7f3f1819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 --2- 172.21.15.3:0/2193495415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f18074350 0x7f3f1819a1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 >> 172.21.15.3:0/2193495415 conn(0x7f3f180fd440 msgr2=0x7f3f180fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 shutdown_connections 2024-08-01T04:15:06.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.317+0000 7f3f1f200640 1 -- 172.21.15.3:0/2193495415 wait complete. 2024-08-01T04:15:06.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:06 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3889879219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-01T04:15:06.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:06 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3889879219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-01T04:15:06.553 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:06 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3889879219' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-01T04:15:07.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 -- 172.21.15.3:0/2588341414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a01020a0 msgr2=0x7f00a010e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 --2- 172.21.15.3:0/2588341414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a01020a0 0x7f00a010e5d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0098009ca0 tx=0x7f009802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 -- 172.21.15.3:0/2588341414 shutdown_connections 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 --2- 172.21.15.3:0/2588341414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a01020a0 0x7f00a010e5d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 --2- 172.21.15.3:0/2588341414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a0101700 0x7f00a0101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 --2- 172.21.15.3:0/2588341414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00a0107700 0x7f00a0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 -- 172.21.15.3:0/2588341414 >> 172.21.15.3:0/2588341414 conn(0x7f00a00fd440 msgr2=0x7f00a00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 -- 172.21.15.3:0/2588341414 shutdown_connections 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.924+0000 7f009fe00640 1 -- 172.21.15.3:0/2588341414 wait complete. 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 Processor -- start 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 -- start start 2024-08-01T04:15:07.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00a0101700 0x7f00a019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 0x7f00a019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0107700 0x7f00a019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111b00 con 0x7f00a01020a0 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111980 con 0x7f00a0107700 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.926+0000 7f009fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0111c80 con 0x7f00a0101700 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 0x7f00a019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 0x7f00a019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51840/0 (socket says 172.21.15.3:51840) 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 -- 172.21.15.3:0/338328446 learned_addr learned my addr 172.21.15.3:0/338328446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009f400640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0107700 0x7f00a019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009ea00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00a0101700 0x7f00a019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 -- 172.21.15.3:0/338328446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00a0101700 msgr2=0x7f00a019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00a0101700 0x7f00a019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 -- 172.21.15.3:0/338328446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0107700 msgr2=0x7f00a019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0107700 0x7f00a019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.927+0000 7f009e000640 1 -- 172.21.15.3:0/338328446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a019f140 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.928+0000 7f009e000640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 0x7f00a019a6d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f008c00ea60 tx=0x7f008c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.928+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c00cde0 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.928+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f008c0035f0 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.928+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a019f430 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.928+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c010670 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.928+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a01a6d10 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.929+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a0101b60 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.931+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f008c0037a0 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.933+0000 7f0097400640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f00840a2c10 0x7f00840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.933+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f008c0cf500 con 0x7f00a01020a0 2024-08-01T04:15:07.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.933+0000 7f009ea00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f00840a2c10 0x7f00840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:07.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.934+0000 7f009ea00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f00840a2c10 0x7f00840a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0090009b50 tx=0x7f0090005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:07.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:06.937+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f008c09c280 con 0x7f00a01020a0 2024-08-01T04:15:07.145 INFO:teuthology.orchestra.run.smithi003.stdout:335007449116 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.138+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f00a01a7040 con 0x7f00a01020a0 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.138+0000 7f0097400640 1 -- 172.21.15.3:0/338328446 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f008c09bb80 con 0x7f00a01020a0 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.141+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f00840a2c10 msgr2=0x7f00840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.141+0000 7f009fe00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f00840a2c10 0x7f00840a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0090009b50 tx=0x7f0090005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.141+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 msgr2=0x7f00a019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.141+0000 7f009fe00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 0x7f00a019a6d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f008c00ea60 tx=0x7f008c00ef20 comp rx=0 tx=0).stop 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 shutdown_connections 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f00840a2c10 0x7f00840a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0107700 0x7f00a019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a01020a0 0x7f00a019a6d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 --2- 172.21.15.3:0/338328446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00a0101700 0x7f00a019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 >> 172.21.15.3:0/338328446 conn(0x7f00a00fd440 msgr2=0x7f00a010cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 shutdown_connections 2024-08-01T04:15:07.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.142+0000 7f009fe00640 1 -- 172.21.15.3:0/338328446 wait complete. 2024-08-01T04:15:07.373 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:07 smithi003 ceph-mon[32262]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:07.373 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:07 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2193495415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-01T04:15:07.373 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:07 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/338328446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-01T04:15:07.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:07 smithi163 ceph-mon[37383]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:07.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:07 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2193495415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-01T04:15:07.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:07 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/338328446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-01T04:15:07.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:07 smithi160 ceph-mon[37224]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:07.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:07 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2193495415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-01T04:15:07.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:07 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/338328446' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-01T04:15:08.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:08 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2276664802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-01T04:15:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 -- 172.21.15.3:0/2883887592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108b30 msgr2=0x7f649c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2883887592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108b30 0x7f649c10cb30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6488009900 tx=0x7f648802a410 comp rx=0 tx=0).stop 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 -- 172.21.15.3:0/2883887592 shutdown_connections 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2883887592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 0x7f649c114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2883887592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108b30 0x7f649c10cb30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2883887592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f649c108180 0x7f649c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 -- 172.21.15.3:0/2883887592 >> 172.21.15.3:0/2883887592 conn(0x7f649c0778d0 msgr2=0x7f649c079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 -- 172.21.15.3:0/2883887592 shutdown_connections 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.395+0000 7f64a3c00640 1 -- 172.21.15.3:0/2883887592 wait complete. 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.396+0000 7f64a3c00640 1 Processor -- start 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 -- start start 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 0x7f649c1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f649c108b30 0x7f649c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 0x7f649c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1182b0 con 0x7f649c108180 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f649c118130 con 0x7f649c108b30 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a3c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f649c118430 con 0x7f649c10d260 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 0x7f649c1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 0x7f649c1a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51862/0 (socket says 172.21.15.3:51862) 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a1200640 1 -- 172.21.15.3:0/2276664802 learned_addr learned my addr 172.21.15.3:0/2276664802 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f649be00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f649c108b30 0x7f649c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.397+0000 7f64a1c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 0x7f649c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1200640 1 -- 172.21.15.3:0/2276664802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 msgr2=0x7f649c19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1200640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 0x7f649c19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1200640 1 -- 172.21.15.3:0/2276664802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f649c108b30 msgr2=0x7f649c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1200640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f649c108b30 0x7f649c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1200640 1 -- 172.21.15.3:0/2276664802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c19ecf0 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 0x7f649c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a1200640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 0x7f649c1a4400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f649000e9f0 tx=0x7f649000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649000cd80 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c19efe0 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6490010910 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.398+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490018990 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.399+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c1ab2d0 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.400+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c108560 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.402+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6490010ac0 con 0x7f649c108180 2024-08-01T04:15:08.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.404+0000 7f6499600640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f64840a2c10 0x7f64840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.405+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f64900cf610 con 0x7f649c108180 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.405+0000 7f649be00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f64840a2c10 0x7f64840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.405+0000 7f649be00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f64840a2c10 0x7f64840a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6488000c00 tx=0x7f6488005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.409+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f649009c390 con 0x7f649c108180 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.611+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f649c19faf0 con 0x7f649c108180 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.611+0000 7f6499600640 1 -- 172.21.15.3:0/2276664802 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f649009bc90 con 0x7f649c108180 2024-08-01T04:15:08.505 INFO:teuthology.orchestra.run.smithi003.stdout:304942678049 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.614+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f64840a2c10 msgr2=0x7f64840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.614+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f64840a2c10 0x7f64840a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6488000c00 tx=0x7f6488005b50 comp rx=0 tx=0).stop 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.614+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 msgr2=0x7f649c1a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.614+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 0x7f649c1a4400 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f649000e9f0 tx=0x7f649000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 shutdown_connections 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f64840a2c10 0x7f64840a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f649c10d260 0x7f649c19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f649c108b30 0x7f649c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 --2- 172.21.15.3:0/2276664802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f649c108180 0x7f649c1a4400 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 >> 172.21.15.3:0/2276664802 conn(0x7f649c0778d0 msgr2=0x7f649c078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 shutdown_connections 2024-08-01T04:15:08.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:07.615+0000 7f64a3c00640 1 -- 172.21.15.3:0/2276664802 wait complete. 2024-08-01T04:15:08.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:08 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2276664802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 -- 172.21.15.3:0/410580316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c108b30 msgr2=0x7f4b3c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 --2- 172.21.15.3:0/410580316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c108b30 0x7f4b3c10cb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 -- 172.21.15.3:0/410580316 shutdown_connections 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 --2- 172.21.15.3:0/410580316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c10d260 0x7f4b3c114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 --2- 172.21.15.3:0/410580316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c108b30 0x7f4b3c10cb30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 --2- 172.21.15.3:0/410580316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 -- 172.21.15.3:0/410580316 >> 172.21.15.3:0/410580316 conn(0x7f4b3c0778d0 msgr2=0x7f4b3c079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 -- 172.21.15.3:0/410580316 shutdown_connections 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.540+0000 7f4b44000640 1 -- 172.21.15.3:0/410580316 wait complete. 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.541+0000 7f4b44000640 1 Processor -- start 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.541+0000 7f4b44000640 1 -- start start 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b44000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c1a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b44000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c108b30 0x7f4b3c1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b44000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c10d260 0x7f4b3c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b44000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1182b0 con 0x7f4b3c108180 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b44000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c118130 con 0x7f4b3c10d260 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b44000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c118430 con 0x7f4b3c108b30 2024-08-01T04:15:08.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b41600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c1a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b41600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c1a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51880/0 (socket says 172.21.15.3:51880) 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b41600640 1 -- 172.21.15.3:0/3490951544 learned_addr learned my addr 172.21.15.3:0/3490951544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b40c00640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c108b30 0x7f4b3c1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.542+0000 7f4b42000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c10d260 0x7f4b3c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b41600640 1 -- 172.21.15.3:0/3490951544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c108b30 msgr2=0x7f4b3c1a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b41600640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c108b30 0x7f4b3c1a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b41600640 1 -- 172.21.15.3:0/3490951544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c10d260 msgr2=0x7f4b3c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b41600640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c10d260 0x7f4b3c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b41600640 1 -- 172.21.15.3:0/3490951544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c19ed30 con 0x7f4b3c108180 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b40c00640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c108b30 0x7f4b3c1a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b41600640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c1a4440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000e9f0 tx=0x7f4b3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:08.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000cd80 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b30010910 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c19f020 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30018990 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.543+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1ab310 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.545+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c108560 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.547+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4b30010ac0 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.548+0000 7f4b3a000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b240a2c10 0x7f4b240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.548+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f4b300cf610 con 0x7f4b3c108180 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.548+0000 7f4b40c00640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b240a2c10 0x7f4b240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.549+0000 7f4b40c00640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b240a2c10 0x7f4b240a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:08.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.552+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4b3009c390 con 0x7f4b3c108180 2024-08-01T04:15:08.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.753+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4b3c110500 con 0x7f4b3c108180 2024-08-01T04:15:08.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.754+0000 7f4b3a000640 1 -- 172.21.15.3:0/3490951544 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4b3009bc90 con 0x7f4b3c108180 2024-08-01T04:15:08.756 INFO:teuthology.orchestra.run.smithi003.stdout:163208757307 2024-08-01T04:15:08.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.757+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b240a2c10 msgr2=0x7f4b240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.757+0000 7f4b44000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b240a2c10 0x7f4b240a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c000c00 tx=0x7f4b2c005a40 comp rx=0 tx=0).stop 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.757+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 msgr2=0x7f4b3c1a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.757+0000 7f4b44000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c1a4440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000e9f0 tx=0x7f4b3000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 shutdown_connections 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f4b240a2c10 0x7f4b240a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b3c10d260 0x7f4b3c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b3c108b30 0x7f4b3c1a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 --2- 172.21.15.3:0/3490951544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4b3c108180 0x7f4b3c1a4440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 >> 172.21.15.3:0/3490951544 conn(0x7f4b3c0778d0 msgr2=0x7f4b3c078bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 shutdown_connections 2024-08-01T04:15:08.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:08.758+0000 7f4b44000640 1 -- 172.21.15.3:0/3490951544 wait complete. 2024-08-01T04:15:08.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:08 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2276664802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 -- 172.21.15.3:0/1594761752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4107700 msgr2=0x7fb6d4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/1594761752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4107700 0x7fb6d4107ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009930 tx=0x7fb6c402a420 comp rx=0 tx=0).stop 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 -- 172.21.15.3:0/1594761752 shutdown_connections 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/1594761752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6d41020a0 0x7fb6d410e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/1594761752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4101700 0x7fb6d4101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/1594761752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4107700 0x7fb6d4107ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 -- 172.21.15.3:0/1594761752 >> 172.21.15.3:0/1594761752 conn(0x7fb6d40fd440 msgr2=0x7fb6d40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:09.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.321+0000 7fb6d8a00640 1 -- 172.21.15.3:0/1594761752 shutdown_connections 2024-08-01T04:15:09.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.322+0000 7fb6d8a00640 1 -- 172.21.15.3:0/1594761752 wait complete. 2024-08-01T04:15:09.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 Processor -- start 2024-08-01T04:15:09.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 -- start start 2024-08-01T04:15:09.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4101700 0x7fb6d419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:09.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6d41020a0 0x7fb6d419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:09.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 0x7fb6d419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:09.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113d50 con 0x7fb6d41020a0 2024-08-01T04:15:09.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113bd0 con 0x7fb6d4107700 2024-08-01T04:15:09.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.323+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113ed0 con 0x7fb6d4101700 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 0x7fb6d419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 0x7fb6d419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:38650/0 (socket says 172.21.15.3:38650) 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 -- 172.21.15.3:0/4103076202 learned_addr learned my addr 172.21.15.3:0/4103076202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3400640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4101700 0x7fb6d419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d2a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6d41020a0 0x7fb6d419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 -- 172.21.15.3:0/4103076202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4101700 msgr2=0x7fb6d419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:09.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4101700 0x7fb6d419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 -- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6d41020a0 msgr2=0x7fb6d419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:09.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.324+0000 7fb6d3e00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6d41020a0 0x7fb6d419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.325+0000 7fb6d3e00640 1 -- 172.21.15.3:0/4103076202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d419f190 con 0x7fb6d4107700 2024-08-01T04:15:09.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.325+0000 7fb6d3e00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 0x7fb6d419ea90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009fd0 tx=0x7fb6cc00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:09.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.325+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc017070 con 0x7fb6d4107700 2024-08-01T04:15:09.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.325+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d419f480 con 0x7fb6d4107700 2024-08-01T04:15:09.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.326+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d406b940 con 0x7fb6d4107700 2024-08-01T04:15:09.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.326+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6cc003e50 con 0x7fb6d4107700 2024-08-01T04:15:09.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.327+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc010460 con 0x7fb6d4107700 2024-08-01T04:15:09.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.327+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b4005290 con 0x7fb6d4107700 2024-08-01T04:15:09.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.330+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb6cc004000 con 0x7fb6d4107700 2024-08-01T04:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.331+0000 7fb6cbe00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb6b80a2c10 0x7fb6b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.332+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fb6cc0cf330 con 0x7fb6d4107700 2024-08-01T04:15:09.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.332+0000 7fb6d3400640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb6b80a2c10 0x7fb6b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:09.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.333+0000 7fb6d3400640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb6b80a2c10 0x7fb6b80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006010 tx=0x7fb6c4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:09.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.333+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb6cc09c0b0 con 0x7fb6d4107700 2024-08-01T04:15:09.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:09 smithi003 ceph-mon[32262]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:09.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:09 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3490951544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-01T04:15:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.533+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb6b4005850 con 0x7fb6d4107700 2024-08-01T04:15:09.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.534+0000 7fb6cbe00640 1 -- 172.21.15.3:0/4103076202 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb6cc09b9b0 con 0x7fb6d4107700 2024-08-01T04:15:09.536 INFO:teuthology.orchestra.run.smithi003.stdout:197568495669 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.536+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb6b80a2c10 msgr2=0x7fb6b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.536+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb6b80a2c10 0x7fb6b80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006010 tx=0x7fb6c4005a20 comp rx=0 tx=0).stop 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.536+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 msgr2=0x7fb6d419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.536+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 0x7fb6d419ea90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009fd0 tx=0x7fb6cc00b260 comp rx=0 tx=0).stop 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 shutdown_connections 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fb6b80a2c10 0x7fb6b80a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6d4107700 0x7fb6d419ea90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6d41020a0 0x7fb6d419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 --2- 172.21.15.3:0/4103076202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6d4101700 0x7fb6d419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 >> 172.21.15.3:0/4103076202 conn(0x7fb6d40fd440 msgr2=0x7fb6d40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 shutdown_connections 2024-08-01T04:15:09.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:09.537+0000 7fb6d8a00640 1 -- 172.21.15.3:0/4103076202 wait complete. 2024-08-01T04:15:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:09 smithi160 ceph-mon[37224]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:09.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:09 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3490951544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-01T04:15:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:09 smithi163 ceph-mon[37383]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:09 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3490951544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-01T04:15:10.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182468 for osd.3 2024-08-01T04:15:10.092 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:10.563 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:10 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/4103076202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-01T04:15:10.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678047 got 304942678049 for osd.10 2024-08-01T04:15:10.629 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:10 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/4103076202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-01T04:15:10.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:10 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/4103076202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-01T04:15:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 -- 172.21.15.3:0/3050122973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec100742f0 msgr2=0x7fec100746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 --2- 172.21.15.3:0/3050122973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec100742f0 0x7fec100746d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7febfc009900 tx=0x7febfc02a480 comp rx=0 tx=0).stop 2024-08-01T04:15:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 -- 172.21.15.3:0/3050122973 shutdown_connections 2024-08-01T04:15:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 --2- 172.21.15.3:0/3050122973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1010e150 0x7fec10110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 --2- 172.21.15.3:0/3050122973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec10074c10 0x7fec1010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 --2- 172.21.15.3:0/3050122973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec100742f0 0x7fec100746d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 -- 172.21.15.3:0/3050122973 >> 172.21.15.3:0/3050122973 conn(0x7fec100fd610 msgr2=0x7fec100ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:11.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 -- 172.21.15.3:0/3050122973 shutdown_connections 2024-08-01T04:15:11.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.008+0000 7fec18400640 1 -- 172.21.15.3:0/3050122973 wait complete. 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.009+0000 7fec18400640 1 Processor -- start 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.009+0000 7fec18400640 1 -- start start 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec18400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec100742f0 0x7fec1019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec18400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 0x7fec1019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec18400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1010e150 0x7fec1019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec18400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec10113ef0 con 0x7fec1010e150 2024-08-01T04:15:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec18400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fec10113d70 con 0x7fec100742f0 2024-08-01T04:15:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec18400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec10114070 con 0x7fec10074c10 2024-08-01T04:15:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec15a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec100742f0 0x7fec1019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec15000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 0x7fec1019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec15a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec100742f0 0x7fec1019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:38668/0 (socket says 172.21.15.3:38668) 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec15000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 0x7fec1019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:46838/0 (socket says 172.21.15.3:46838) 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec15a00640 1 -- 172.21.15.3:0/3790158395 learned_addr learned my addr 172.21.15.3:0/3790158395 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.010+0000 7fec16400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1010e150 0x7fec1019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec15000640 1 -- 172.21.15.3:0/3790158395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec100742f0 msgr2=0x7fec1019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec15000640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec100742f0 0x7fec1019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec15000640 1 -- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1010e150 msgr2=0x7fec1019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec15000640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1010e150 0x7fec1019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec15000640 1 -- 172.21.15.3:0/3790158395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1019f360 con 0x7fec10074c10 2024-08-01T04:15:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec15000640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 0x7fec1019a8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9f0 tx=0x7fec0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.011+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0400ccc0 con 0x7fec10074c10 2024-08-01T04:15:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.012+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1019f650 con 0x7fec10074c10 2024-08-01T04:15:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.012+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec04010910 con 0x7fec10074c10 2024-08-01T04:15:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.012+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec04018960 con 0x7fec10074c10 2024-08-01T04:15:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.012+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec101a6f30 con 0x7fec10074c10 2024-08-01T04:15:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.013+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec100746d0 con 0x7fec10074c10 2024-08-01T04:15:11.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.016+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fec04010ac0 con 0x7fec10074c10 2024-08-01T04:15:11.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.017+0000 7fec0e000640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7febf80a2c10 0x7febf80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.017+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fec040cf690 con 0x7fec10074c10 2024-08-01T04:15:11.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.018+0000 7fec15a00640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7febf80a2c10 0x7febf80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.019+0000 7fec15a00640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7febf80a2c10 0x7febf80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febfc0096c0 tx=0x7febfc0046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:11.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.021+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fec0409c410 con 0x7fec10074c10 2024-08-01T04:15:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.248+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fec1002d5e0 con 0x7fec10074c10 2024-08-01T04:15:11.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.249+0000 7fec0e000640 1 -- 172.21.15.3:0/3790158395 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fec0409bd10 con 0x7fec10074c10 2024-08-01T04:15:11.251 INFO:teuthology.orchestra.run.smithi003.stdout:141733920833 2024-08-01T04:15:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7febf80a2c10 msgr2=0x7febf80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7febf80a2c10 0x7febf80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febfc0096c0 tx=0x7febfc0046c0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 msgr2=0x7fec1019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 0x7fec1019a8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9f0 tx=0x7fec0400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 shutdown_connections 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7febf80a2c10 0x7febf80a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec1010e150 0x7fec1019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec10074c10 0x7fec1019a8f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 --2- 172.21.15.3:0/3790158395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec100742f0 0x7fec1019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 >> 172.21.15.3:0/3790158395 conn(0x7fec100fd610 msgr2=0x7fec100fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:11.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 shutdown_connections 2024-08-01T04:15:11.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.251+0000 7fec18400640 1 -- 172.21.15.3:0/3790158395 wait complete. 2024-08-01T04:15:11.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:11 smithi003 ceph-mon[32262]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:11.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:11 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3790158395' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-01T04:15:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:11 smithi160 ceph-mon[37224]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:11 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3790158395' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-01T04:15:11.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:11 smithi163 ceph-mon[37383]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:11 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3790158395' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 -- 172.21.15.3:0/1825540142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 msgr2=0x7fad84107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 --2- 172.21.15.3:0/1825540142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 0x7fad84107ad0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fad74009be0 tx=0x7fad7402a6c0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 -- 172.21.15.3:0/1825540142 shutdown_connections 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 --2- 172.21.15.3:0/1825540142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad84102090 0x7fad8410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 --2- 172.21.15.3:0/1825540142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad841016f0 0x7fad84101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 --2- 172.21.15.3:0/1825540142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 0x7fad84107ad0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.969+0000 7fad83200640 1 -- 172.21.15.3:0/1825540142 >> 172.21.15.3:0/1825540142 conn(0x7fad840fd470 msgr2=0x7fad840ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.970+0000 7fad83200640 1 -- 172.21.15.3:0/1825540142 shutdown_connections 2024-08-01T04:15:11.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.970+0000 7fad83200640 1 -- 172.21.15.3:0/1825540142 wait complete. 2024-08-01T04:15:11.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.971+0000 7fad83200640 1 Processor -- start 2024-08-01T04:15:11.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.971+0000 7fad83200640 1 -- start start 2024-08-01T04:15:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad83200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad841016f0 0x7fad84074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad83200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84102090 0x7fad84074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad83200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 0x7fad84074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad83200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad84111ab0 con 0x7fad84102090 2024-08-01T04:15:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad83200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fad84111930 con 0x7fad841076f0 2024-08-01T04:15:11.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad83200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fad84111c30 con 0x7fad841016f0 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad81400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84102090 0x7fad84074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad81e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad841016f0 0x7fad84074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad81400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84102090 0x7fad84074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60142/0 (socket says 172.21.15.3:60142) 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad81400640 1 -- 172.21.15.3:0/4003378035 learned_addr learned my addr 172.21.15.3:0/4003378035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.972+0000 7fad82800640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 0x7fad84074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.973+0000 7fad81e00640 1 -- 172.21.15.3:0/4003378035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 msgr2=0x7fad84074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.973+0000 7fad81e00640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 0x7fad84074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.973+0000 7fad81e00640 1 -- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84102090 msgr2=0x7fad84074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.973+0000 7fad81e00640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84102090 0x7fad84074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.973+0000 7fad81e00640 1 -- 172.21.15.3:0/4003378035 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad841a6450 con 0x7fad841016f0 2024-08-01T04:15:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.973+0000 7fad81e00640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad841016f0 0x7fad84074220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad7402abe0 tx=0x7fad74030c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.974+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74041020 con 0x7fad841016f0 2024-08-01T04:15:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.974+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad841a66e0 con 0x7fad841016f0 2024-08-01T04:15:11.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.974+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad74003e50 con 0x7fad841016f0 2024-08-01T04:15:11.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.974+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74036040 con 0x7fad841016f0 2024-08-01T04:15:11.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.974+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad841a6be0 con 0x7fad841016f0 2024-08-01T04:15:11.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.975+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad84101b50 con 0x7fad841016f0 2024-08-01T04:15:11.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.977+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fad74044020 con 0x7fad841016f0 2024-08-01T04:15:11.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.978+0000 7fad7aa00640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fad680a2c90 0x7fad680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:11.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.979+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fad740efd10 con 0x7fad841016f0 2024-08-01T04:15:11.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.979+0000 7fad81400640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fad680a2c90 0x7fad680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:11.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.980+0000 7fad81400640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fad680a2c90 0x7fad680a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad84075cb0 tx=0x7fad6c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:11.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:11.983+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fad740f5020 con 0x7fad841016f0 2024-08-01T04:15:12.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.181+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fad84109c60 con 0x7fad841016f0 2024-08-01T04:15:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.182+0000 7fad7aa00640 1 -- 172.21.15.3:0/4003378035 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fad7404b020 con 0x7fad841016f0 2024-08-01T04:15:12.184 INFO:teuthology.orchestra.run.smithi003.stdout:219043332145 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.184+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fad680a2c90 msgr2=0x7fad680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.184+0000 7fad83200640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fad680a2c90 0x7fad680a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad84075cb0 tx=0x7fad6c009260 comp rx=0 tx=0).stop 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.184+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad841016f0 msgr2=0x7fad84074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.184+0000 7fad83200640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad841016f0 0x7fad84074220 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad7402abe0 tx=0x7fad74030c50 comp rx=0 tx=0).stop 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 shutdown_connections 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fad680a2c90 0x7fad680a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad841076f0 0x7fad84074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad84102090 0x7fad84074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 --2- 172.21.15.3:0/4003378035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad841016f0 0x7fad84074220 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 >> 172.21.15.3:0/4003378035 conn(0x7fad840fd470 msgr2=0x7fad840fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 shutdown_connections 2024-08-01T04:15:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:12.185+0000 7fad83200640 1 -- 172.21.15.3:0/4003378035 wait complete. 2024-08-01T04:15:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:12 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/4003378035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-01T04:15:12.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:12 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/4003378035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-01T04:15:12.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:12 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/4003378035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-01T04:15:12.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738451 for osd.0 2024-08-01T04:15:12.948 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:13.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495669 for osd.6 2024-08-01T04:15:13.091 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:13.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:13 smithi003 ceph-mon[32262]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:13.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:13 smithi160 ceph-mon[37224]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:13.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:13 smithi163 ceph-mon[37383]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:14.402 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757305 got 163208757307 for osd.5 2024-08-01T04:15:14.403 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:14.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 -- 172.21.15.3:0/2610572359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c103260 msgr2=0x7fc85c10b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:14.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 --2- 172.21.15.3:0/2610572359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c103260 0x7fc85c10b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc848009900 tx=0x7fc84802a480 comp rx=0 tx=0).stop 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 -- 172.21.15.3:0/2610572359 shutdown_connections 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 --2- 172.21.15.3:0/2610572359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c10bd20 0x7fc85c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 --2- 172.21.15.3:0/2610572359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c103260 0x7fc85c10b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 --2- 172.21.15.3:0/2610572359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 0x7fc85c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 -- 172.21.15.3:0/2610572359 >> 172.21.15.3:0/2610572359 conn(0x7fc85c0fc340 msgr2=0x7fc85c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 -- 172.21.15.3:0/2610572359 shutdown_connections 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.634+0000 7fc862000640 1 -- 172.21.15.3:0/2610572359 wait complete. 2024-08-01T04:15:14.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.635+0000 7fc862000640 1 Processor -- start 2024-08-01T04:15:14.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 -- start start 2024-08-01T04:15:14.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 0x7fc85c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:14.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 0x7fc85c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:14.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c10bd20 0x7fc85c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:14.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c111b40 con 0x7fc85c102940 2024-08-01T04:15:14.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c1119c0 con 0x7fc85c103260 2024-08-01T04:15:14.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.636+0000 7fc862000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c111cc0 con 0x7fc85c10bd20 2024-08-01T04:15:14.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 0x7fc85c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 0x7fc85c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34640/0 (socket says 172.21.15.3:34640) 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 -- 172.21.15.3:0/1888904560 learned_addr learned my addr 172.21.15.3:0/1888904560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc860c00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 0x7fc85c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc861600640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c10bd20 0x7fc85c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 -- 172.21.15.3:0/1888904560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c10bd20 msgr2=0x7fc85c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c10bd20 0x7fc85c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 -- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 msgr2=0x7fc85c19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.637+0000 7fc85be00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 0x7fc85c19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.638+0000 7fc85be00640 1 -- 172.21.15.3:0/1888904560 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c19f1a0 con 0x7fc85c103260 2024-08-01T04:15:14.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.638+0000 7fc860c00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 0x7fc85c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:14.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.638+0000 7fc85be00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 0x7fc85c19a730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc848004770 tx=0x7fc848003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:14.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.639+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84803f070 con 0x7fc85c103260 2024-08-01T04:15:14.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.639+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc848031ca0 con 0x7fc85c103260 2024-08-01T04:15:14.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.639+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848035030 con 0x7fc85c103260 2024-08-01T04:15:14.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.639+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c19f430 con 0x7fc85c103260 2024-08-01T04:15:14.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.639+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c1a6d70 con 0x7fc85c103260 2024-08-01T04:15:14.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.640+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c102d20 con 0x7fc85c103260 2024-08-01T04:15:14.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.642+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc848042070 con 0x7fc85c103260 2024-08-01T04:15:14.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.646+0000 7fc859600640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8440a2c90 0x7fc8440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:14.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.646+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fc8480ef430 con 0x7fc85c103260 2024-08-01T04:15:14.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.646+0000 7fc860c00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8440a2c90 0x7fc8440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:14.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.647+0000 7fc860c00640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8440a2c90 0x7fc8440a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc850009fc0 tx=0x7fc850005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:14.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.647+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc8480bbc20 con 0x7fc85c103260 2024-08-01T04:15:14.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332145 for osd.7 2024-08-01T04:15:14.674 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.856+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc85c1a6f40 con 0x7fc85c103260 2024-08-01T04:15:14.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.857+0000 7fc859600640 1 -- 172.21.15.3:0/1888904560 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc8480a6090 con 0x7fc85c103260 2024-08-01T04:15:14.859 INFO:teuthology.orchestra.run.smithi003.stdout:77309411405 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.860+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8440a2c90 msgr2=0x7fc8440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.860+0000 7fc862000640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8440a2c90 0x7fc8440a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc850009fc0 tx=0x7fc850005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.860+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 msgr2=0x7fc85c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.860+0000 7fc862000640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 0x7fc85c19a730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc848004770 tx=0x7fc848003c10 comp rx=0 tx=0).stop 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 shutdown_connections 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8440a2c90 0x7fc8440a5150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc85c10bd20 0x7fc85c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc85c103260 0x7fc85c19a730 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 --2- 172.21.15.3:0/1888904560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc85c102940 0x7fc85c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:14.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 >> 172.21.15.3:0/1888904560 conn(0x7fc85c0fc340 msgr2=0x7fc85c10d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:14.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 shutdown_connections 2024-08-01T04:15:14.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:14.861+0000 7fc862000640 1 -- 172.21.15.3:0/1888904560 wait complete. 2024-08-01T04:15:15.369 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920833 for osd.4 2024-08-01T04:15:15.369 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:15.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:15 smithi003 ceph-mon[32262]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:15.393 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:15 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1888904560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-01T04:15:15.598 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:15 smithi163 ceph-mon[37383]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:15.598 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:15 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1888904560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-01T04:15:15.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:15 smithi160 ceph-mon[37224]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:15.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:15 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1888904560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-01T04:15:15.854 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411405 for osd.2 2024-08-01T04:15:15.854 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:16.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449112 got 335007449116 for osd.11 2024-08-01T04:15:16.021 DEBUG:teuthology.parallel:result is None 2024-08-01T04:15:16.021 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-01T04:15:16.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:15:16.515 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 -- 172.21.15.3:0/344053884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074240 msgr2=0x7f16b8074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 --2- 172.21.15.3:0/344053884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074240 0x7f16b8074620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009900 tx=0x7f16a802a480 comp rx=0 tx=0).stop 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 -- 172.21.15.3:0/344053884 shutdown_connections 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 --2- 172.21.15.3:0/344053884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b810dfb0 0x7f16b81103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 --2- 172.21.15.3:0/344053884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074b60 0x7f16b810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 --2- 172.21.15.3:0/344053884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074240 0x7f16b8074620 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.165+0000 7f16bca00640 1 -- 172.21.15.3:0/344053884 >> 172.21.15.3:0/344053884 conn(0x7f16b80fd470 msgr2=0x7f16b80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:17.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.166+0000 7f16bca00640 1 -- 172.21.15.3:0/344053884 shutdown_connections 2024-08-01T04:15:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.166+0000 7f16bca00640 1 -- 172.21.15.3:0/344053884 wait complete. 2024-08-01T04:15:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.167+0000 7f16bca00640 1 Processor -- start 2024-08-01T04:15:17.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 -- start start 2024-08-01T04:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 0x7f16b819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 0x7f16b819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b810dfb0 0x7f16b819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8113d00 con 0x7f16b810dfb0 2024-08-01T04:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8113b80 con 0x7f16b8074240 2024-08-01T04:15:17.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16bca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8113e80 con 0x7f16b8074b60 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16b7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 0x7f16b819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16b6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 0x7f16b819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16b6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 0x7f16b819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53186/0 (socket says 172.21.15.3:53186) 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16b6a00640 1 -- 172.21.15.3:0/3953228788 learned_addr learned my addr 172.21.15.3:0/3953228788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.168+0000 7f16b7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 0x7f16b819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34652/0 (socket says 172.21.15.3:34652) 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b7e00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b810dfb0 0x7f16b819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:17.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b6a00640 1 -- 172.21.15.3:0/3953228788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 msgr2=0x7f16b819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b6a00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 0x7f16b819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b6a00640 1 -- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b810dfb0 msgr2=0x7f16b819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b6a00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b810dfb0 0x7f16b819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b6a00640 1 -- 172.21.15.3:0/3953228788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b819f230 con 0x7f16b8074b60 2024-08-01T04:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b7400640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 0x7f16b819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:17.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.169+0000 7f16b6a00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 0x7f16b819a7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e9f0 tx=0x7f16a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.170+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400cd80 con 0x7f16b8074b60 2024-08-01T04:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.170+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4010910 con 0x7f16b8074b60 2024-08-01T04:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.170+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4018990 con 0x7f16b8074b60 2024-08-01T04:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.170+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b819f520 con 0x7f16b8074b60 2024-08-01T04:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.170+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b81a6e00 con 0x7f16b8074b60 2024-08-01T04:15:17.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.172+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b8074620 con 0x7f16b8074b60 2024-08-01T04:15:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.176+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f16a4010ac0 con 0x7f16b8074b60 2024-08-01T04:15:17.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.178+0000 7f16afe00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f169c0a2c90 0x7f169c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.178+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f16a40cf920 con 0x7f16b8074b60 2024-08-01T04:15:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.178+0000 7f16b7400640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f169c0a2c90 0x7f169c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:17.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.179+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f16a409c050 con 0x7f16b8074b60 2024-08-01T04:15:17.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.179+0000 7f16b7400640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f169c0a2c90 0x7f169c0a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16a8000c00 tx=0x7f16a8031510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:17.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:17.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:17 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:17 smithi003 ceph-mon[32262]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-01T04:15:17.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.372+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16b8109160 con 0x7f169c0a2c90 2024-08-01T04:15:17.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.388+0000 7f16afe00640 1 -- 172.21.15.3:0/3953228788 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258923 (secure 0 0 0) 0x7f16b8109160 con 0x7f169c0a2c90 2024-08-01T04:15:17.392 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f169c0a2c90 msgr2=0x7f169c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f169c0a2c90 0x7f169c0a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16a8000c00 tx=0x7f16a8031510 comp rx=0 tx=0).stop 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 msgr2=0x7f16b819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 0x7f16b819a7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e9f0 tx=0x7f16a400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 shutdown_connections 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f169c0a2c90 0x7f169c0a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b810dfb0 0x7f16b819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16b8074b60 0x7f16b819a7c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 --2- 172.21.15.3:0/3953228788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16b8074240 0x7f16b819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 >> 172.21.15.3:0/3953228788 conn(0x7f16b80fd470 msgr2=0x7f16b80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 shutdown_connections 2024-08-01T04:15:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:17.394+0000 7f16bca00640 1 -- 172.21.15.3:0/3953228788 wait complete. 2024-08-01T04:15:17.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:17 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:17.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:17 smithi163 ceph-mon[37383]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-01T04:15:17.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:17.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:17 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:17.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:17 smithi160 ceph-mon[37224]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-01T04:15:18.377 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-08-01T04:15:16.214966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657326,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":548,"num_read_kb":395,"num_write":349,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365128,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576696,"statfs":{"total":1151940427776,"available":1151566536704,"internally_reserved":0,"allocated":12005376,"data_stored":7576146,"data_compressed":176910,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004251"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029836+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:14:50.029836+0000","last_peered":"2024-08-01T04:14:50.029836+0000","last_clean":"2024-08-01T04:14:50.029836+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:14:50.029836+0000","last_undegraded":"2024-08-01T04:14:50.029836+0000","last_fullsized":"2024-08-01T04:14:50.029836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.044564+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:14:50.044564+0000","last_peered":"2024-08-01T04:14:50.044564+0000","last_clean":"2024-08-01T04:14:50.044564+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:14:50.044564+0000","last_undegraded":"2024-08-01T04:14:50.044564+0000","last_fullsized":"2024-08-01T04:14:50.044564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.047120+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:14:50.047120+0000","last_peered":"2024-08-01T04:14:50.047120+0000","last_clean":"2024-08-01T04:14:50.047120+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:14:50.047120+0000","last_undegraded":"2024-08-01T04:14:50.047120+0000","last_fullsized":"2024-08-01T04:14:50.047120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.045288+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:14:50.045288+0000","last_peered":"2024-08-01T04:14:50.045288+0000","last_clean":"2024-08-01T04:14:50.045288+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:14:50.045288+0000","last_undegraded":"2024-08-01T04:14:50.045288+0000","last_fullsized":"2024-08-01T04:14:50.045288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032217+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:14:50.032217+0000","last_peered":"2024-08-01T04:14:50.032217+0000","last_clean":"2024-08-01T04:14:50.032217+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:14:50.032217+0000","last_undegraded":"2024-08-01T04:14:50.032217+0000","last_fullsized":"2024-08-01T04:14:50.032217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029385+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:14:50.029385+0000","last_peered":"2024-08-01T04:14:50.029385+0000","last_clean":"2024-08-01T04:14:50.029385+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:14:50.029385+0000","last_undegraded":"2024-08-01T04:14:50.029385+0000","last_fullsized":"2024-08-01T04:14:50.029385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.030928+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:14:50.030928+0000","last_peered":"2024-08-01T04:14:50.030928+0000","last_clean":"2024-08-01T04:14:50.030928+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:14:50.030928+0000","last_undegraded":"2024-08-01T04:14:50.030928+0000","last_fullsized":"2024-08-01T04:14:50.030928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.037974+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:14:50.037974+0000","last_peered":"2024-08-01T04:14:50.037974+0000","last_clean":"2024-08-01T04:14:50.037974+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:14:50.037974+0000","last_undegraded":"2024-08-01T04:14:50.037974+0000","last_fullsized":"2024-08-01T04:14:50.037974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033482+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:14:50.033482+0000","last_peered":"2024-08-01T04:14:50.033482+0000","last_clean":"2024-08-01T04:14:50.033482+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:14:50.033482+0000","last_undegraded":"2024-08-01T04:14:50.033482+0000","last_fullsized":"2024-08-01T04:14:50.033482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029958+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:14:50.029958+0000","last_peered":"2024-08-01T04:14:50.029958+0000","last_clean":"2024-08-01T04:14:50.029958+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:14:50.029958+0000","last_undegraded":"2024-08-01T04:14:50.029958+0000","last_fullsized":"2024-08-01T04:14:50.029958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031900+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:14:50.031900+0000","last_peered":"2024-08-01T04:14:50.031900+0000","last_clean":"2024-08-01T04:14:50.031900+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:14:50.031900+0000","last_undegraded":"2024-08-01T04:14:50.031900+0000","last_fullsized":"2024-08-01T04:14:50.031900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.040347+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:14:50.040347+0000","last_peered":"2024-08-01T04:14:50.040347+0000","last_clean":"2024-08-01T04:14:50.040347+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:14:50.040347+0000","last_undegraded":"2024-08-01T04:14:50.040347+0000","last_fullsized":"2024-08-01T04:14:50.040347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042581+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:14:50.042581+0000","last_peered":"2024-08-01T04:14:50.042581+0000","last_clean":"2024-08-01T04:14:50.042581+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:14:50.042581+0000","last_undegraded":"2024-08-01T04:14:50.042581+0000","last_fullsized":"2024-08-01T04:14:50.042581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046083+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:14:50.046083+0000","last_peered":"2024-08-01T04:14:50.046083+0000","last_clean":"2024-08-01T04:14:50.046083+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:14:50.046083+0000","last_undegraded":"2024-08-01T04:14:50.046083+0000","last_fullsized":"2024-08-01T04:14:50.046083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046996+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:14:50.046996+0000","last_peered":"2024-08-01T04:14:50.046996+0000","last_clean":"2024-08-01T04:14:50.046996+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:14:50.046996+0000","last_undegraded":"2024-08-01T04:14:50.046996+0000","last_fullsized":"2024-08-01T04:14:50.046996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032735+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:14:50.032735+0000","last_peered":"2024-08-01T04:14:50.032735+0000","last_clean":"2024-08-01T04:14:50.032735+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:14:50.032735+0000","last_undegraded":"2024-08-01T04:14:50.032735+0000","last_fullsized":"2024-08-01T04:14:50.032735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029909+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:14:50.029909+0000","last_peered":"2024-08-01T04:14:50.029909+0000","last_clean":"2024-08-01T04:14:50.029909+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:14:50.029909+0000","last_undegraded":"2024-08-01T04:14:50.029909+0000","last_fullsized":"2024-08-01T04:14:50.029909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032159+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:14:50.032159+0000","last_peered":"2024-08-01T04:14:50.032159+0000","last_clean":"2024-08-01T04:14:50.032159+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:14:50.032159+0000","last_undegraded":"2024-08-01T04:14:50.032159+0000","last_fullsized":"2024-08-01T04:14:50.032159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032744+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:14:50.032744+0000","last_peered":"2024-08-01T04:14:50.032744+0000","last_clean":"2024-08-01T04:14:50.032744+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:14:50.032744+0000","last_undegraded":"2024-08-01T04:14:50.032744+0000","last_fullsized":"2024-08-01T04:14:50.032744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029656+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:14:50.029656+0000","last_peered":"2024-08-01T04:14:50.029656+0000","last_clean":"2024-08-01T04:14:50.029656+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:14:50.029656+0000","last_undegraded":"2024-08-01T04:14:50.029656+0000","last_fullsized":"2024-08-01T04:14:50.029656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042709+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:14:50.042709+0000","last_peered":"2024-08-01T04:14:50.042709+0000","last_clean":"2024-08-01T04:14:50.042709+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:14:50.042709+0000","last_undegraded":"2024-08-01T04:14:50.042709+0000","last_fullsized":"2024-08-01T04:14:50.042709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.043134+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:14:50.043134+0000","last_peered":"2024-08-01T04:14:50.043134+0000","last_clean":"2024-08-01T04:14:50.043134+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:14:50.043134+0000","last_undegraded":"2024-08-01T04:14:50.043134+0000","last_fullsized":"2024-08-01T04:14:50.043134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042859+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:14:50.042859+0000","last_peered":"2024-08-01T04:14:50.042859+0000","last_clean":"2024-08-01T04:14:50.042859+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:14:50.042859+0000","last_undegraded":"2024-08-01T04:14:50.042859+0000","last_fullsized":"2024-08-01T04:14:50.042859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.047767+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:14:50.047767+0000","last_peered":"2024-08-01T04:14:50.047767+0000","last_clean":"2024-08-01T04:14:50.047767+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:14:50.047767+0000","last_undegraded":"2024-08-01T04:14:50.047767+0000","last_fullsized":"2024-08-01T04:14:50.047767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042865+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:14:50.042865+0000","last_peered":"2024-08-01T04:14:50.042865+0000","last_clean":"2024-08-01T04:14:50.042865+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:14:50.042865+0000","last_undegraded":"2024-08-01T04:14:50.042865+0000","last_fullsized":"2024-08-01T04:14:50.042865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033056+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:14:50.033056+0000","last_peered":"2024-08-01T04:14:50.033056+0000","last_clean":"2024-08-01T04:14:50.033056+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:14:50.033056+0000","last_undegraded":"2024-08-01T04:14:50.033056+0000","last_fullsized":"2024-08-01T04:14:50.033056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029761+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:14:50.029761+0000","last_peered":"2024-08-01T04:14:50.029761+0000","last_clean":"2024-08-01T04:14:50.029761+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:14:50.029761+0000","last_undegraded":"2024-08-01T04:14:50.029761+0000","last_fullsized":"2024-08-01T04:14:50.029761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041046+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:14:50.041046+0000","last_peered":"2024-08-01T04:14:50.041046+0000","last_clean":"2024-08-01T04:14:50.041046+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:14:50.041046+0000","last_undegraded":"2024-08-01T04:14:50.041046+0000","last_fullsized":"2024-08-01T04:14:50.041046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.047405+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:14:50.047405+0000","last_peered":"2024-08-01T04:14:50.047405+0000","last_clean":"2024-08-01T04:14:50.047405+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:14:50.047405+0000","last_undegraded":"2024-08-01T04:14:50.047405+0000","last_fullsized":"2024-08-01T04:14:50.047405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.052215+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:14:50.052215+0000","last_peered":"2024-08-01T04:14:50.052215+0000","last_clean":"2024-08-01T04:14:50.052215+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:14:50.052215+0000","last_undegraded":"2024-08-01T04:14:50.052215+0000","last_fullsized":"2024-08-01T04:14:50.052215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032005+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:14:50.032005+0000","last_peered":"2024-08-01T04:14:50.032005+0000","last_clean":"2024-08-01T04:14:50.032005+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:14:50.032005+0000","last_undegraded":"2024-08-01T04:14:50.032005+0000","last_fullsized":"2024-08-01T04:14:50.032005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029466+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:14:50.029466+0000","last_peered":"2024-08-01T04:14:50.029466+0000","last_clean":"2024-08-01T04:14:50.029466+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:14:50.029466+0000","last_undegraded":"2024-08-01T04:14:50.029466+0000","last_fullsized":"2024-08-01T04:14:50.029466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032484+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:14:50.032484+0000","last_peered":"2024-08-01T04:14:50.032484+0000","last_clean":"2024-08-01T04:14:50.032484+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:14:50.032484+0000","last_undegraded":"2024-08-01T04:14:50.032484+0000","last_fullsized":"2024-08-01T04:14:50.032484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:12.729611+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:15:12.729611+0000","last_peered":"2024-08-01T04:15:12.729611+0000","last_clean":"2024-08-01T04:15:12.729611+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:15:12.729611+0000","last_undegraded":"2024-08-01T04:15:12.729611+0000","last_fullsized":"2024-08-01T04:15:12.729611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035470+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:14:50.035470+0000","last_peered":"2024-08-01T04:14:50.035470+0000","last_clean":"2024-08-01T04:14:50.035470+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:14:50.035470+0000","last_undegraded":"2024-08-01T04:14:50.035470+0000","last_fullsized":"2024-08-01T04:14:50.035470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.048326+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:14:50.048326+0000","last_peered":"2024-08-01T04:14:50.048326+0000","last_clean":"2024-08-01T04:14:50.048326+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:14:50.048326+0000","last_undegraded":"2024-08-01T04:14:50.048326+0000","last_fullsized":"2024-08-01T04:14:50.048326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041883+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:14:50.041883+0000","last_peered":"2024-08-01T04:14:50.041883+0000","last_clean":"2024-08-01T04:14:50.041883+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:14:50.041883+0000","last_undegraded":"2024-08-01T04:14:50.041883+0000","last_fullsized":"2024-08-01T04:14:50.041883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.038275+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:14:50.038275+0000","last_peered":"2024-08-01T04:14:50.038275+0000","last_clean":"2024-08-01T04:14:50.038275+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:14:50.038275+0000","last_undegraded":"2024-08-01T04:14:50.038275+0000","last_fullsized":"2024-08-01T04:14:50.038275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035317+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:14:50.035317+0000","last_peered":"2024-08-01T04:14:50.035317+0000","last_clean":"2024-08-01T04:14:50.035317+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:14:50.035317+0000","last_undegraded":"2024-08-01T04:14:50.035317+0000","last_fullsized":"2024-08-01T04:14:50.035317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031122+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:14:50.031122+0000","last_peered":"2024-08-01T04:14:50.031122+0000","last_clean":"2024-08-01T04:14:50.031122+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:14:50.031122+0000","last_undegraded":"2024-08-01T04:14:50.031122+0000","last_fullsized":"2024-08-01T04:14:50.031122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:10.726005+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:15:10.726005+0000","last_peered":"2024-08-01T04:15:10.726005+0000","last_clean":"2024-08-01T04:15:10.726005+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:15:10.726005+0000","last_undegraded":"2024-08-01T04:15:10.726005+0000","last_fullsized":"2024-08-01T04:15:10.726005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033248+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:14:50.033248+0000","last_peered":"2024-08-01T04:14:50.033248+0000","last_clean":"2024-08-01T04:14:50.033248+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:14:50.033248+0000","last_undegraded":"2024-08-01T04:14:50.033248+0000","last_fullsized":"2024-08-01T04:14:50.033248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031957+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:14:50.031957+0000","last_peered":"2024-08-01T04:14:50.031957+0000","last_clean":"2024-08-01T04:14:50.031957+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:14:50.031957+0000","last_undegraded":"2024-08-01T04:14:50.031957+0000","last_fullsized":"2024-08-01T04:14:50.031957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046813+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:14:50.046813+0000","last_peered":"2024-08-01T04:14:50.046813+0000","last_clean":"2024-08-01T04:14:50.046813+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:14:50.046813+0000","last_undegraded":"2024-08-01T04:14:50.046813+0000","last_fullsized":"2024-08-01T04:14:50.046813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035595+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:14:50.035595+0000","last_peered":"2024-08-01T04:14:50.035595+0000","last_clean":"2024-08-01T04:14:50.035595+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:14:50.035595+0000","last_undegraded":"2024-08-01T04:14:50.035595+0000","last_fullsized":"2024-08-01T04:14:50.035595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.044245+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:14:50.044245+0000","last_peered":"2024-08-01T04:14:50.044245+0000","last_clean":"2024-08-01T04:14:50.044245+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:14:50.044245+0000","last_undegraded":"2024-08-01T04:14:50.044245+0000","last_fullsized":"2024-08-01T04:14:50.044245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":188,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:11.727387+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:15:11.727387+0000","last_peered":"2024-08-01T04:15:11.727387+0000","last_clean":"2024-08-01T04:15:11.727387+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:15:11.727387+0000","last_undegraded":"2024-08-01T04:15:11.727387+0000","last_fullsized":"2024-08-01T04:15:11.727387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62334,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":127,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032228+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:14:50.032228+0000","last_peered":"2024-08-01T04:14:50.032228+0000","last_clean":"2024-08-01T04:14:50.032228+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:14:50.032228+0000","last_undegraded":"2024-08-01T04:14:50.032228+0000","last_fullsized":"2024-08-01T04:14:50.032228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032613+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:14:50.032613+0000","last_peered":"2024-08-01T04:14:50.032613+0000","last_clean":"2024-08-01T04:14:50.032613+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:14:50.032613+0000","last_undegraded":"2024-08-01T04:14:50.032613+0000","last_fullsized":"2024-08-01T04:14:50.032613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041795+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:14:50.041795+0000","last_peered":"2024-08-01T04:14:50.041795+0000","last_clean":"2024-08-01T04:14:50.041795+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:14:50.041795+0000","last_undegraded":"2024-08-01T04:14:50.041795+0000","last_fullsized":"2024-08-01T04:14:50.041795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035226+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:14:50.035226+0000","last_peered":"2024-08-01T04:14:50.035226+0000","last_clean":"2024-08-01T04:14:50.035226+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:14:50.035226+0000","last_undegraded":"2024-08-01T04:14:50.035226+0000","last_fullsized":"2024-08-01T04:14:50.035226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031666+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:14:50.031666+0000","last_peered":"2024-08-01T04:14:50.031666+0000","last_clean":"2024-08-01T04:14:50.031666+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:14:50.031666+0000","last_undegraded":"2024-08-01T04:14:50.031666+0000","last_fullsized":"2024-08-01T04:14:50.031666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:11.743829+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:15:11.743829+0000","last_peered":"2024-08-01T04:15:11.743829+0000","last_clean":"2024-08-01T04:15:11.743829+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:15:11.743829+0000","last_undegraded":"2024-08-01T04:15:11.743829+0000","last_fullsized":"2024-08-01T04:15:11.743829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.044294+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:14:50.044294+0000","last_peered":"2024-08-01T04:14:50.044294+0000","last_clean":"2024-08-01T04:14:50.044294+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:14:50.044294+0000","last_undegraded":"2024-08-01T04:14:50.044294+0000","last_fullsized":"2024-08-01T04:14:50.044294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032017+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:14:50.032017+0000","last_peered":"2024-08-01T04:14:50.032017+0000","last_clean":"2024-08-01T04:14:50.032017+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:14:50.032017+0000","last_undegraded":"2024-08-01T04:14:50.032017+0000","last_fullsized":"2024-08-01T04:14:50.032017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035144+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:14:50.035144+0000","last_peered":"2024-08-01T04:14:50.035144+0000","last_clean":"2024-08-01T04:14:50.035144+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:14:50.035144+0000","last_undegraded":"2024-08-01T04:14:50.035144+0000","last_fullsized":"2024-08-01T04:14:50.035144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032382+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:14:50.032382+0000","last_peered":"2024-08-01T04:14:50.032382+0000","last_clean":"2024-08-01T04:14:50.032382+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:14:50.032382+0000","last_undegraded":"2024-08-01T04:14:50.032382+0000","last_fullsized":"2024-08-01T04:14:50.032382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029831+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:14:50.029831+0000","last_peered":"2024-08-01T04:14:50.029831+0000","last_clean":"2024-08-01T04:14:50.029831+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:14:50.029831+0000","last_undegraded":"2024-08-01T04:14:50.029831+0000","last_fullsized":"2024-08-01T04:14:50.029831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041105+0000","last_change":"2024-08-01T04:13:51.466500+0000","last_active":"2024-08-01T04:14:50.041105+0000","last_peered":"2024-08-01T04:14:50.041105+0000","last_clean":"2024-08-01T04:14:50.041105+0000","last_became_active":"2024-08-01T04:13:51.466069+0000","last_became_peered":"2024-08-01T04:13:51.466069+0000","last_unstale":"2024-08-01T04:14:50.041105+0000","last_undegraded":"2024-08-01T04:14:50.041105+0000","last_fullsized":"2024-08-01T04:14:50.041105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:30:03.961179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032101+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:14:50.032101+0000","last_peered":"2024-08-01T04:14:50.032101+0000","last_clean":"2024-08-01T04:14:50.032101+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:14:50.032101+0000","last_undegraded":"2024-08-01T04:14:50.032101+0000","last_fullsized":"2024-08-01T04:14:50.032101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035723+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:14:50.035723+0000","last_peered":"2024-08-01T04:14:50.035723+0000","last_clean":"2024-08-01T04:14:50.035723+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:14:50.035723+0000","last_undegraded":"2024-08-01T04:14:50.035723+0000","last_fullsized":"2024-08-01T04:14:50.035723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031481+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:14:50.031481+0000","last_peered":"2024-08-01T04:14:50.031481+0000","last_clean":"2024-08-01T04:14:50.031481+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:14:50.031481+0000","last_undegraded":"2024-08-01T04:14:50.031481+0000","last_fullsized":"2024-08-01T04:14:50.031481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033154+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:14:50.033154+0000","last_peered":"2024-08-01T04:14:50.033154+0000","last_clean":"2024-08-01T04:14:50.033154+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:14:50.033154+0000","last_undegraded":"2024-08-01T04:14:50.033154+0000","last_fullsized":"2024-08-01T04:14:50.033154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032021+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:14:50.032021+0000","last_peered":"2024-08-01T04:14:50.032021+0000","last_clean":"2024-08-01T04:14:50.032021+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:14:50.032021+0000","last_undegraded":"2024-08-01T04:14:50.032021+0000","last_fullsized":"2024-08-01T04:14:50.032021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031484+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:14:50.031484+0000","last_peered":"2024-08-01T04:14:50.031484+0000","last_clean":"2024-08-01T04:14:50.031484+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:14:50.031484+0000","last_undegraded":"2024-08-01T04:14:50.031484+0000","last_fullsized":"2024-08-01T04:14:50.031484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.039870+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:14:50.039870+0000","last_peered":"2024-08-01T04:14:50.039870+0000","last_clean":"2024-08-01T04:14:50.039870+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:14:50.039870+0000","last_undegraded":"2024-08-01T04:14:50.039870+0000","last_fullsized":"2024-08-01T04:14:50.039870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041066+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:14:50.041066+0000","last_peered":"2024-08-01T04:14:50.041066+0000","last_clean":"2024-08-01T04:14:50.041066+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:14:50.041066+0000","last_undegraded":"2024-08-01T04:14:50.041066+0000","last_fullsized":"2024-08-01T04:14:50.041066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031367+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:14:50.031367+0000","last_peered":"2024-08-01T04:14:50.031367+0000","last_clean":"2024-08-01T04:14:50.031367+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:14:50.031367+0000","last_undegraded":"2024-08-01T04:14:50.031367+0000","last_fullsized":"2024-08-01T04:14:50.031367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041790+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:14:50.041790+0000","last_peered":"2024-08-01T04:14:50.041790+0000","last_clean":"2024-08-01T04:14:50.041790+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:14:50.041790+0000","last_undegraded":"2024-08-01T04:14:50.041790+0000","last_fullsized":"2024-08-01T04:14:50.041790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:13.747169+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:15:13.747169+0000","last_peered":"2024-08-01T04:15:13.747169+0000","last_clean":"2024-08-01T04:15:13.747169+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:15:13.747169+0000","last_undegraded":"2024-08-01T04:15:13.747169+0000","last_fullsized":"2024-08-01T04:15:13.747169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035183+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:14:50.035183+0000","last_peered":"2024-08-01T04:14:50.035183+0000","last_clean":"2024-08-01T04:14:50.035183+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:14:50.035183+0000","last_undegraded":"2024-08-01T04:14:50.035183+0000","last_fullsized":"2024-08-01T04:14:50.035183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031391+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:14:50.031391+0000","last_peered":"2024-08-01T04:14:50.031391+0000","last_clean":"2024-08-01T04:14:50.031391+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:14:50.031391+0000","last_undegraded":"2024-08-01T04:14:50.031391+0000","last_fullsized":"2024-08-01T04:14:50.031391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029513+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:14:50.029513+0000","last_peered":"2024-08-01T04:14:50.029513+0000","last_clean":"2024-08-01T04:14:50.029513+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:14:50.029513+0000","last_undegraded":"2024-08-01T04:14:50.029513+0000","last_fullsized":"2024-08-01T04:14:50.029513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032882+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:14:50.032882+0000","last_peered":"2024-08-01T04:14:50.032882+0000","last_clean":"2024-08-01T04:14:50.032882+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:14:50.032882+0000","last_undegraded":"2024-08-01T04:14:50.032882+0000","last_fullsized":"2024-08-01T04:14:50.032882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029710+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:14:50.029710+0000","last_peered":"2024-08-01T04:14:50.029710+0000","last_clean":"2024-08-01T04:14:50.029710+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:14:50.029710+0000","last_undegraded":"2024-08-01T04:14:50.029710+0000","last_fullsized":"2024-08-01T04:14:50.029710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032718+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:14:50.032718+0000","last_peered":"2024-08-01T04:14:50.032718+0000","last_clean":"2024-08-01T04:14:50.032718+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:14:50.032718+0000","last_undegraded":"2024-08-01T04:14:50.032718+0000","last_fullsized":"2024-08-01T04:14:50.032718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041070+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:14:50.041070+0000","last_peered":"2024-08-01T04:14:50.041070+0000","last_clean":"2024-08-01T04:14:50.041070+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:14:50.041070+0000","last_undegraded":"2024-08-01T04:14:50.041070+0000","last_fullsized":"2024-08-01T04:14:50.041070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032616+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:14:50.032616+0000","last_peered":"2024-08-01T04:14:50.032616+0000","last_clean":"2024-08-01T04:14:50.032616+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:14:50.032616+0000","last_undegraded":"2024-08-01T04:14:50.032616+0000","last_fullsized":"2024-08-01T04:14:50.032616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042981+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:14:50.042981+0000","last_peered":"2024-08-01T04:14:50.042981+0000","last_clean":"2024-08-01T04:14:50.042981+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:14:50.042981+0000","last_undegraded":"2024-08-01T04:14:50.042981+0000","last_fullsized":"2024-08-01T04:14:50.042981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.040400+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:14:50.040400+0000","last_peered":"2024-08-01T04:14:50.040400+0000","last_clean":"2024-08-01T04:14:50.040400+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:14:50.040400+0000","last_undegraded":"2024-08-01T04:14:50.040400+0000","last_fullsized":"2024-08-01T04:14:50.040400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.051995+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:14:50.051995+0000","last_peered":"2024-08-01T04:14:50.051995+0000","last_clean":"2024-08-01T04:14:50.051995+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:14:50.051995+0000","last_undegraded":"2024-08-01T04:14:50.051995+0000","last_fullsized":"2024-08-01T04:14:50.051995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032827+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:14:50.032827+0000","last_peered":"2024-08-01T04:14:50.032827+0000","last_clean":"2024-08-01T04:14:50.032827+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:14:50.032827+0000","last_undegraded":"2024-08-01T04:14:50.032827+0000","last_fullsized":"2024-08-01T04:14:50.032827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032073+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:14:50.032073+0000","last_peered":"2024-08-01T04:14:50.032073+0000","last_clean":"2024-08-01T04:14:50.032073+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:14:50.032073+0000","last_undegraded":"2024-08-01T04:14:50.032073+0000","last_fullsized":"2024-08-01T04:14:50.032073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.043063+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:14:50.043063+0000","last_peered":"2024-08-01T04:14:50.043063+0000","last_clean":"2024-08-01T04:14:50.043063+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:14:50.043063+0000","last_undegraded":"2024-08-01T04:14:50.043063+0000","last_fullsized":"2024-08-01T04:14:50.043063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029561+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:14:50.029561+0000","last_peered":"2024-08-01T04:14:50.029561+0000","last_clean":"2024-08-01T04:14:50.029561+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:14:50.029561+0000","last_undegraded":"2024-08-01T04:14:50.029561+0000","last_fullsized":"2024-08-01T04:14:50.029561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.052459+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:14:50.052459+0000","last_peered":"2024-08-01T04:14:50.052459+0000","last_clean":"2024-08-01T04:14:50.052459+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:14:50.052459+0000","last_undegraded":"2024-08-01T04:14:50.052459+0000","last_fullsized":"2024-08-01T04:14:50.052459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.038977+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:14:50.038977+0000","last_peered":"2024-08-01T04:14:50.038977+0000","last_clean":"2024-08-01T04:14:50.038977+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:14:50.038977+0000","last_undegraded":"2024-08-01T04:14:50.038977+0000","last_fullsized":"2024-08-01T04:14:50.038977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032324+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:14:50.032324+0000","last_peered":"2024-08-01T04:14:50.032324+0000","last_clean":"2024-08-01T04:14:50.032324+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:14:50.032324+0000","last_undegraded":"2024-08-01T04:14:50.032324+0000","last_fullsized":"2024-08-01T04:14:50.032324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031199+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:14:50.031199+0000","last_peered":"2024-08-01T04:14:50.031199+0000","last_clean":"2024-08-01T04:14:50.031199+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:14:50.031199+0000","last_undegraded":"2024-08-01T04:14:50.031199+0000","last_fullsized":"2024-08-01T04:14:50.031199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.037426+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:14:50.037426+0000","last_peered":"2024-08-01T04:14:50.037426+0000","last_clean":"2024-08-01T04:14:50.037426+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:14:50.037426+0000","last_undegraded":"2024-08-01T04:14:50.037426+0000","last_fullsized":"2024-08-01T04:14:50.037426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032792+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:14:50.032792+0000","last_peered":"2024-08-01T04:14:50.032792+0000","last_clean":"2024-08-01T04:14:50.032792+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:14:50.032792+0000","last_undegraded":"2024-08-01T04:14:50.032792+0000","last_fullsized":"2024-08-01T04:14:50.032792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032849+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:14:50.032849+0000","last_peered":"2024-08-01T04:14:50.032849+0000","last_clean":"2024-08-01T04:14:50.032849+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:14:50.032849+0000","last_undegraded":"2024-08-01T04:14:50.032849+0000","last_fullsized":"2024-08-01T04:14:50.032849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032537+0000","last_change":"2024-08-01T04:13:51.461259+0000","last_active":"2024-08-01T04:14:50.032537+0000","last_peered":"2024-08-01T04:14:50.032537+0000","last_clean":"2024-08-01T04:14:50.032537+0000","last_became_active":"2024-08-01T04:13:51.460884+0000","last_became_peered":"2024-08-01T04:13:51.460884+0000","last_unstale":"2024-08-01T04:14:50.032537+0000","last_undegraded":"2024-08-01T04:14:50.032537+0000","last_fullsized":"2024-08-01T04:14:50.032537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:18.454778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046684+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:14:50.046684+0000","last_peered":"2024-08-01T04:14:50.046684+0000","last_clean":"2024-08-01T04:14:50.046684+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:14:50.046684+0000","last_undegraded":"2024-08-01T04:14:50.046684+0000","last_fullsized":"2024-08-01T04:14:50.046684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042436+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:14:50.042436+0000","last_peered":"2024-08-01T04:14:50.042436+0000","last_clean":"2024-08-01T04:14:50.042436+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:14:50.042436+0000","last_undegraded":"2024-08-01T04:14:50.042436+0000","last_fullsized":"2024-08-01T04:14:50.042436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041784+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:14:50.041784+0000","last_peered":"2024-08-01T04:14:50.041784+0000","last_clean":"2024-08-01T04:14:50.041784+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:14:50.041784+0000","last_undegraded":"2024-08-01T04:14:50.041784+0000","last_fullsized":"2024-08-01T04:14:50.041784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033142+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:14:50.033142+0000","last_peered":"2024-08-01T04:14:50.033142+0000","last_clean":"2024-08-01T04:14:50.033142+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:14:50.033142+0000","last_undegraded":"2024-08-01T04:14:50.033142+0000","last_fullsized":"2024-08-01T04:14:50.033142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66724,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":412,"num_read_kb":276,"num_write":166,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211962,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":467596,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1121801,"data_compressed":41648,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841575,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":467832,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":467752,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1060071,"data_compressed":41648,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":467221,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":1059365,"data_compressed":41648,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":468048,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":467532,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531088,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":529348,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":468492,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:15:18.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:15:18.451 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:15:18.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:15:18.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:18 smithi003 ceph-mon[32262]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:15:18.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:18 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:15:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:15:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:18 smithi163 ceph-mon[37383]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:15:18.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:18 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:15:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:15:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:18 smithi160 ceph-mon[37224]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:15:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:18 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' 2024-08-01T04:15:18.878 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:19.290 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:19 smithi003 ceph-mon[32262]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-01T04:15:19.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.536+0000 7f5355a00640 1 -- 172.21.15.3:0/1433997472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 msgr2=0x7f5350101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:19.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.536+0000 7f5355a00640 1 --2- 172.21.15.3:0/1433997472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350101b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5338009890 tx=0x7f533802a360 comp rx=0 tx=0).stop 2024-08-01T04:15:19.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 -- 172.21.15.3:0/1433997472 shutdown_connections 2024-08-01T04:15:19.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 --2- 172.21.15.3:0/1433997472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5350102090 0x7f535010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 --2- 172.21.15.3:0/1433997472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350101b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 --2- 172.21.15.3:0/1433997472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53501076f0 0x7f5350107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 -- 172.21.15.3:0/1433997472 >> 172.21.15.3:0/1433997472 conn(0x7f53500fd470 msgr2=0x7f53500ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 -- 172.21.15.3:0/1433997472 shutdown_connections 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.537+0000 7f5355a00640 1 -- 172.21.15.3:0/1433997472 wait complete. 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.538+0000 7f5355a00640 1 Processor -- start 2024-08-01T04:15:19.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.539+0000 7f5355a00640 1 -- start start 2024-08-01T04:15:19.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.539+0000 7f5355a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:19.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.539+0000 7f5355a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5350102090 0x7f5350074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:19.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f5355a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53501076f0 0x7f5350074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:19.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f5355a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111af0 con 0x7f53501076f0 2024-08-01T04:15:19.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f5355a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111970 con 0x7f5350102090 2024-08-01T04:15:19.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f5355a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5350111c70 con 0x7f53501016f0 2024-08-01T04:15:19.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53202/0 (socket says 172.21.15.3:53202) 2024-08-01T04:15:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534fe00640 1 -- 172.21.15.3:0/3271354845 learned_addr learned my addr 172.21.15.3:0/3271354845 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534f400640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5350102090 0x7f5350074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f5355000640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53501076f0 0x7f5350074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:19.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534f400640 1 -- 172.21.15.3:0/3271354845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 msgr2=0x7f5350074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534f400640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534f400640 1 -- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53501076f0 msgr2=0x7f5350074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534f400640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53501076f0 0x7f5350074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.540+0000 7f534f400640 1 -- 172.21.15.3:0/3271354845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53501a64d0 con 0x7f5350102090 2024-08-01T04:15:19.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.541+0000 7f534f400640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5350102090 0x7f5350074760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5338035040 tx=0x7f5338033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.541+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338037040 con 0x7f5350102090 2024-08-01T04:15:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.541+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5338033d90 con 0x7f5350102090 2024-08-01T04:15:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.541+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338003830 con 0x7f5350102090 2024-08-01T04:15:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.541+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53501a6760 con 0x7f5350102090 2024-08-01T04:15:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.541+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53501a6c90 con 0x7f5350102090 2024-08-01T04:15:19.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.542+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5350101b50 con 0x7f5350102090 2024-08-01T04:15:19.548 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:19 smithi163 ceph-mon[37383]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-01T04:15:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.545+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5338042030 con 0x7f5350102090 2024-08-01T04:15:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.546+0000 7f534cc00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f53340a2c90 0x7f53340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.547+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f53380eec20 con 0x7f5350102090 2024-08-01T04:15:19.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.547+0000 7f534fe00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f53340a2c90 0x7f53340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.548+0000 7f534fe00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f53340a2c90 0x7f53340a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5348009a60 tx=0x7f5348010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:19.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.548+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f53380bbb40 con 0x7f5350102090 2024-08-01T04:15:19.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:19 smithi160 ceph-mon[37224]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-01T04:15:19.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.742+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5350062360 con 0x7f53340a2c90 2024-08-01T04:15:19.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.754+0000 7f534cc00640 1 -- 172.21.15.3:0/3271354845 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258924 (secure 0 0 0) 0x7f5350062360 con 0x7f53340a2c90 2024-08-01T04:15:19.760 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:19.761 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:15:19.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f53340a2c90 msgr2=0x7f53340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:19.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f53340a2c90 0x7f53340a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5348009a60 tx=0x7f5348010040 comp rx=0 tx=0).stop 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5350102090 msgr2=0x7f5350074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5350102090 0x7f5350074760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5338035040 tx=0x7f5338033ae0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 shutdown_connections 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f53340a2c90 0x7f53340a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53501076f0 0x7f5350074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5350102090 0x7f5350074760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 --2- 172.21.15.3:0/3271354845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53501016f0 0x7f5350074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 >> 172.21.15.3:0/3271354845 conn(0x7f53500fd470 msgr2=0x7f53500fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 shutdown_connections 2024-08-01T04:15:19.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:19.759+0000 7f5355a00640 1 -- 172.21.15.3:0/3271354845 wait complete. 2024-08-01T04:15:20.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:20 smithi163 ceph-mon[37383]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:15:20.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:20 smithi003 ceph-mon[32262]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:15:20.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:20 smithi160 ceph-mon[37224]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:15:20.761 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-08-01T04:15:18.215647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657326,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":568,"num_read_kb":405,"num_write":349,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365128,"kb_used_data":11724,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576696,"statfs":{"total":1151940427776,"available":1151566536704,"internally_reserved":0,"allocated":12005376,"data_stored":7576146,"data_compressed":176910,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":39,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004273"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029836+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:14:50.029836+0000","last_peered":"2024-08-01T04:14:50.029836+0000","last_clean":"2024-08-01T04:14:50.029836+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:14:50.029836+0000","last_undegraded":"2024-08-01T04:14:50.029836+0000","last_fullsized":"2024-08-01T04:14:50.029836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.044564+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:14:50.044564+0000","last_peered":"2024-08-01T04:14:50.044564+0000","last_clean":"2024-08-01T04:14:50.044564+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:14:50.044564+0000","last_undegraded":"2024-08-01T04:14:50.044564+0000","last_fullsized":"2024-08-01T04:14:50.044564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.047120+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:14:50.047120+0000","last_peered":"2024-08-01T04:14:50.047120+0000","last_clean":"2024-08-01T04:14:50.047120+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:14:50.047120+0000","last_undegraded":"2024-08-01T04:14:50.047120+0000","last_fullsized":"2024-08-01T04:14:50.047120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.045288+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:14:50.045288+0000","last_peered":"2024-08-01T04:14:50.045288+0000","last_clean":"2024-08-01T04:14:50.045288+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:14:50.045288+0000","last_undegraded":"2024-08-01T04:14:50.045288+0000","last_fullsized":"2024-08-01T04:14:50.045288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032217+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:14:50.032217+0000","last_peered":"2024-08-01T04:14:50.032217+0000","last_clean":"2024-08-01T04:14:50.032217+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:14:50.032217+0000","last_undegraded":"2024-08-01T04:14:50.032217+0000","last_fullsized":"2024-08-01T04:14:50.032217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029385+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:14:50.029385+0000","last_peered":"2024-08-01T04:14:50.029385+0000","last_clean":"2024-08-01T04:14:50.029385+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:14:50.029385+0000","last_undegraded":"2024-08-01T04:14:50.029385+0000","last_fullsized":"2024-08-01T04:14:50.029385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.030928+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:14:50.030928+0000","last_peered":"2024-08-01T04:14:50.030928+0000","last_clean":"2024-08-01T04:14:50.030928+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:14:50.030928+0000","last_undegraded":"2024-08-01T04:14:50.030928+0000","last_fullsized":"2024-08-01T04:14:50.030928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.037974+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:14:50.037974+0000","last_peered":"2024-08-01T04:14:50.037974+0000","last_clean":"2024-08-01T04:14:50.037974+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:14:50.037974+0000","last_undegraded":"2024-08-01T04:14:50.037974+0000","last_fullsized":"2024-08-01T04:14:50.037974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033482+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:14:50.033482+0000","last_peered":"2024-08-01T04:14:50.033482+0000","last_clean":"2024-08-01T04:14:50.033482+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:14:50.033482+0000","last_undegraded":"2024-08-01T04:14:50.033482+0000","last_fullsized":"2024-08-01T04:14:50.033482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029958+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:14:50.029958+0000","last_peered":"2024-08-01T04:14:50.029958+0000","last_clean":"2024-08-01T04:14:50.029958+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:14:50.029958+0000","last_undegraded":"2024-08-01T04:14:50.029958+0000","last_fullsized":"2024-08-01T04:14:50.029958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031900+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:14:50.031900+0000","last_peered":"2024-08-01T04:14:50.031900+0000","last_clean":"2024-08-01T04:14:50.031900+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:14:50.031900+0000","last_undegraded":"2024-08-01T04:14:50.031900+0000","last_fullsized":"2024-08-01T04:14:50.031900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.040347+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:14:50.040347+0000","last_peered":"2024-08-01T04:14:50.040347+0000","last_clean":"2024-08-01T04:14:50.040347+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:14:50.040347+0000","last_undegraded":"2024-08-01T04:14:50.040347+0000","last_fullsized":"2024-08-01T04:14:50.040347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042581+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:14:50.042581+0000","last_peered":"2024-08-01T04:14:50.042581+0000","last_clean":"2024-08-01T04:14:50.042581+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:14:50.042581+0000","last_undegraded":"2024-08-01T04:14:50.042581+0000","last_fullsized":"2024-08-01T04:14:50.042581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046083+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:14:50.046083+0000","last_peered":"2024-08-01T04:14:50.046083+0000","last_clean":"2024-08-01T04:14:50.046083+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:14:50.046083+0000","last_undegraded":"2024-08-01T04:14:50.046083+0000","last_fullsized":"2024-08-01T04:14:50.046083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046996+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:14:50.046996+0000","last_peered":"2024-08-01T04:14:50.046996+0000","last_clean":"2024-08-01T04:14:50.046996+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:14:50.046996+0000","last_undegraded":"2024-08-01T04:14:50.046996+0000","last_fullsized":"2024-08-01T04:14:50.046996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032735+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:14:50.032735+0000","last_peered":"2024-08-01T04:14:50.032735+0000","last_clean":"2024-08-01T04:14:50.032735+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:14:50.032735+0000","last_undegraded":"2024-08-01T04:14:50.032735+0000","last_fullsized":"2024-08-01T04:14:50.032735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029909+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:14:50.029909+0000","last_peered":"2024-08-01T04:14:50.029909+0000","last_clean":"2024-08-01T04:14:50.029909+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:14:50.029909+0000","last_undegraded":"2024-08-01T04:14:50.029909+0000","last_fullsized":"2024-08-01T04:14:50.029909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032159+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:14:50.032159+0000","last_peered":"2024-08-01T04:14:50.032159+0000","last_clean":"2024-08-01T04:14:50.032159+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:14:50.032159+0000","last_undegraded":"2024-08-01T04:14:50.032159+0000","last_fullsized":"2024-08-01T04:14:50.032159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032744+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:14:50.032744+0000","last_peered":"2024-08-01T04:14:50.032744+0000","last_clean":"2024-08-01T04:14:50.032744+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:14:50.032744+0000","last_undegraded":"2024-08-01T04:14:50.032744+0000","last_fullsized":"2024-08-01T04:14:50.032744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029656+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:14:50.029656+0000","last_peered":"2024-08-01T04:14:50.029656+0000","last_clean":"2024-08-01T04:14:50.029656+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:14:50.029656+0000","last_undegraded":"2024-08-01T04:14:50.029656+0000","last_fullsized":"2024-08-01T04:14:50.029656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042709+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:14:50.042709+0000","last_peered":"2024-08-01T04:14:50.042709+0000","last_clean":"2024-08-01T04:14:50.042709+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:14:50.042709+0000","last_undegraded":"2024-08-01T04:14:50.042709+0000","last_fullsized":"2024-08-01T04:14:50.042709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.043134+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:14:50.043134+0000","last_peered":"2024-08-01T04:14:50.043134+0000","last_clean":"2024-08-01T04:14:50.043134+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:14:50.043134+0000","last_undegraded":"2024-08-01T04:14:50.043134+0000","last_fullsized":"2024-08-01T04:14:50.043134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042859+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:14:50.042859+0000","last_peered":"2024-08-01T04:14:50.042859+0000","last_clean":"2024-08-01T04:14:50.042859+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:14:50.042859+0000","last_undegraded":"2024-08-01T04:14:50.042859+0000","last_fullsized":"2024-08-01T04:14:50.042859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.047767+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:14:50.047767+0000","last_peered":"2024-08-01T04:14:50.047767+0000","last_clean":"2024-08-01T04:14:50.047767+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:14:50.047767+0000","last_undegraded":"2024-08-01T04:14:50.047767+0000","last_fullsized":"2024-08-01T04:14:50.047767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042865+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:14:50.042865+0000","last_peered":"2024-08-01T04:14:50.042865+0000","last_clean":"2024-08-01T04:14:50.042865+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:14:50.042865+0000","last_undegraded":"2024-08-01T04:14:50.042865+0000","last_fullsized":"2024-08-01T04:14:50.042865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033056+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:14:50.033056+0000","last_peered":"2024-08-01T04:14:50.033056+0000","last_clean":"2024-08-01T04:14:50.033056+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:14:50.033056+0000","last_undegraded":"2024-08-01T04:14:50.033056+0000","last_fullsized":"2024-08-01T04:14:50.033056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029761+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:14:50.029761+0000","last_peered":"2024-08-01T04:14:50.029761+0000","last_clean":"2024-08-01T04:14:50.029761+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:14:50.029761+0000","last_undegraded":"2024-08-01T04:14:50.029761+0000","last_fullsized":"2024-08-01T04:14:50.029761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041046+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:14:50.041046+0000","last_peered":"2024-08-01T04:14:50.041046+0000","last_clean":"2024-08-01T04:14:50.041046+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:14:50.041046+0000","last_undegraded":"2024-08-01T04:14:50.041046+0000","last_fullsized":"2024-08-01T04:14:50.041046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.047405+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:14:50.047405+0000","last_peered":"2024-08-01T04:14:50.047405+0000","last_clean":"2024-08-01T04:14:50.047405+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:14:50.047405+0000","last_undegraded":"2024-08-01T04:14:50.047405+0000","last_fullsized":"2024-08-01T04:14:50.047405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.052215+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:14:50.052215+0000","last_peered":"2024-08-01T04:14:50.052215+0000","last_clean":"2024-08-01T04:14:50.052215+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:14:50.052215+0000","last_undegraded":"2024-08-01T04:14:50.052215+0000","last_fullsized":"2024-08-01T04:14:50.052215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032005+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:14:50.032005+0000","last_peered":"2024-08-01T04:14:50.032005+0000","last_clean":"2024-08-01T04:14:50.032005+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:14:50.032005+0000","last_undegraded":"2024-08-01T04:14:50.032005+0000","last_fullsized":"2024-08-01T04:14:50.032005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029466+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:14:50.029466+0000","last_peered":"2024-08-01T04:14:50.029466+0000","last_clean":"2024-08-01T04:14:50.029466+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:14:50.029466+0000","last_undegraded":"2024-08-01T04:14:50.029466+0000","last_fullsized":"2024-08-01T04:14:50.029466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032484+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:14:50.032484+0000","last_peered":"2024-08-01T04:14:50.032484+0000","last_clean":"2024-08-01T04:14:50.032484+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:14:50.032484+0000","last_undegraded":"2024-08-01T04:14:50.032484+0000","last_fullsized":"2024-08-01T04:14:50.032484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:12.729611+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:15:12.729611+0000","last_peered":"2024-08-01T04:15:12.729611+0000","last_clean":"2024-08-01T04:15:12.729611+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:15:12.729611+0000","last_undegraded":"2024-08-01T04:15:12.729611+0000","last_fullsized":"2024-08-01T04:15:12.729611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035470+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:14:50.035470+0000","last_peered":"2024-08-01T04:14:50.035470+0000","last_clean":"2024-08-01T04:14:50.035470+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:14:50.035470+0000","last_undegraded":"2024-08-01T04:14:50.035470+0000","last_fullsized":"2024-08-01T04:14:50.035470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.048326+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:14:50.048326+0000","last_peered":"2024-08-01T04:14:50.048326+0000","last_clean":"2024-08-01T04:14:50.048326+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:14:50.048326+0000","last_undegraded":"2024-08-01T04:14:50.048326+0000","last_fullsized":"2024-08-01T04:14:50.048326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041883+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:14:50.041883+0000","last_peered":"2024-08-01T04:14:50.041883+0000","last_clean":"2024-08-01T04:14:50.041883+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:14:50.041883+0000","last_undegraded":"2024-08-01T04:14:50.041883+0000","last_fullsized":"2024-08-01T04:14:50.041883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.038275+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:14:50.038275+0000","last_peered":"2024-08-01T04:14:50.038275+0000","last_clean":"2024-08-01T04:14:50.038275+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:14:50.038275+0000","last_undegraded":"2024-08-01T04:14:50.038275+0000","last_fullsized":"2024-08-01T04:14:50.038275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035317+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:14:50.035317+0000","last_peered":"2024-08-01T04:14:50.035317+0000","last_clean":"2024-08-01T04:14:50.035317+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:14:50.035317+0000","last_undegraded":"2024-08-01T04:14:50.035317+0000","last_fullsized":"2024-08-01T04:14:50.035317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031122+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:14:50.031122+0000","last_peered":"2024-08-01T04:14:50.031122+0000","last_clean":"2024-08-01T04:14:50.031122+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:14:50.031122+0000","last_undegraded":"2024-08-01T04:14:50.031122+0000","last_fullsized":"2024-08-01T04:14:50.031122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:15.733315+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:15:15.733315+0000","last_peered":"2024-08-01T04:15:15.733315+0000","last_clean":"2024-08-01T04:15:15.733315+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:15:15.733315+0000","last_undegraded":"2024-08-01T04:15:15.733315+0000","last_fullsized":"2024-08-01T04:15:15.733315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033248+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:14:50.033248+0000","last_peered":"2024-08-01T04:14:50.033248+0000","last_clean":"2024-08-01T04:14:50.033248+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:14:50.033248+0000","last_undegraded":"2024-08-01T04:14:50.033248+0000","last_fullsized":"2024-08-01T04:14:50.033248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031957+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:14:50.031957+0000","last_peered":"2024-08-01T04:14:50.031957+0000","last_clean":"2024-08-01T04:14:50.031957+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:14:50.031957+0000","last_undegraded":"2024-08-01T04:14:50.031957+0000","last_fullsized":"2024-08-01T04:14:50.031957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046813+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:14:50.046813+0000","last_peered":"2024-08-01T04:14:50.046813+0000","last_clean":"2024-08-01T04:14:50.046813+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:14:50.046813+0000","last_undegraded":"2024-08-01T04:14:50.046813+0000","last_fullsized":"2024-08-01T04:14:50.046813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035595+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:14:50.035595+0000","last_peered":"2024-08-01T04:14:50.035595+0000","last_clean":"2024-08-01T04:14:50.035595+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:14:50.035595+0000","last_undegraded":"2024-08-01T04:14:50.035595+0000","last_fullsized":"2024-08-01T04:14:50.035595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.044245+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:14:50.044245+0000","last_peered":"2024-08-01T04:14:50.044245+0000","last_clean":"2024-08-01T04:14:50.044245+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:14:50.044245+0000","last_undegraded":"2024-08-01T04:14:50.044245+0000","last_fullsized":"2024-08-01T04:14:50.044245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":198,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:16.734675+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:15:16.734675+0000","last_peered":"2024-08-01T04:15:16.734675+0000","last_clean":"2024-08-01T04:15:16.734675+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:15:16.734675+0000","last_undegraded":"2024-08-01T04:15:16.734675+0000","last_fullsized":"2024-08-01T04:15:16.734675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62334,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":132,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032228+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:14:50.032228+0000","last_peered":"2024-08-01T04:14:50.032228+0000","last_clean":"2024-08-01T04:14:50.032228+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:14:50.032228+0000","last_undegraded":"2024-08-01T04:14:50.032228+0000","last_fullsized":"2024-08-01T04:14:50.032228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032613+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:14:50.032613+0000","last_peered":"2024-08-01T04:14:50.032613+0000","last_clean":"2024-08-01T04:14:50.032613+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:14:50.032613+0000","last_undegraded":"2024-08-01T04:14:50.032613+0000","last_fullsized":"2024-08-01T04:14:50.032613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041795+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:14:50.041795+0000","last_peered":"2024-08-01T04:14:50.041795+0000","last_clean":"2024-08-01T04:14:50.041795+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:14:50.041795+0000","last_undegraded":"2024-08-01T04:14:50.041795+0000","last_fullsized":"2024-08-01T04:14:50.041795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035226+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:14:50.035226+0000","last_peered":"2024-08-01T04:14:50.035226+0000","last_clean":"2024-08-01T04:14:50.035226+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:14:50.035226+0000","last_undegraded":"2024-08-01T04:14:50.035226+0000","last_fullsized":"2024-08-01T04:14:50.035226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031666+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:14:50.031666+0000","last_peered":"2024-08-01T04:14:50.031666+0000","last_clean":"2024-08-01T04:14:50.031666+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:14:50.031666+0000","last_undegraded":"2024-08-01T04:14:50.031666+0000","last_fullsized":"2024-08-01T04:14:50.031666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:16.754168+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:15:16.754168+0000","last_peered":"2024-08-01T04:15:16.754168+0000","last_clean":"2024-08-01T04:15:16.754168+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:15:16.754168+0000","last_undegraded":"2024-08-01T04:15:16.754168+0000","last_fullsized":"2024-08-01T04:15:16.754168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.044294+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:14:50.044294+0000","last_peered":"2024-08-01T04:14:50.044294+0000","last_clean":"2024-08-01T04:14:50.044294+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:14:50.044294+0000","last_undegraded":"2024-08-01T04:14:50.044294+0000","last_fullsized":"2024-08-01T04:14:50.044294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032017+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:14:50.032017+0000","last_peered":"2024-08-01T04:14:50.032017+0000","last_clean":"2024-08-01T04:14:50.032017+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:14:50.032017+0000","last_undegraded":"2024-08-01T04:14:50.032017+0000","last_fullsized":"2024-08-01T04:14:50.032017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035144+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:14:50.035144+0000","last_peered":"2024-08-01T04:14:50.035144+0000","last_clean":"2024-08-01T04:14:50.035144+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:14:50.035144+0000","last_undegraded":"2024-08-01T04:14:50.035144+0000","last_fullsized":"2024-08-01T04:14:50.035144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032382+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:14:50.032382+0000","last_peered":"2024-08-01T04:14:50.032382+0000","last_clean":"2024-08-01T04:14:50.032382+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:14:50.032382+0000","last_undegraded":"2024-08-01T04:14:50.032382+0000","last_fullsized":"2024-08-01T04:14:50.032382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029831+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:14:50.029831+0000","last_peered":"2024-08-01T04:14:50.029831+0000","last_clean":"2024-08-01T04:14:50.029831+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:14:50.029831+0000","last_undegraded":"2024-08-01T04:14:50.029831+0000","last_fullsized":"2024-08-01T04:14:50.029831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041105+0000","last_change":"2024-08-01T04:13:51.466500+0000","last_active":"2024-08-01T04:14:50.041105+0000","last_peered":"2024-08-01T04:14:50.041105+0000","last_clean":"2024-08-01T04:14:50.041105+0000","last_became_active":"2024-08-01T04:13:51.466069+0000","last_became_peered":"2024-08-01T04:13:51.466069+0000","last_unstale":"2024-08-01T04:14:50.041105+0000","last_undegraded":"2024-08-01T04:14:50.041105+0000","last_fullsized":"2024-08-01T04:14:50.041105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:30:03.961179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032101+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:14:50.032101+0000","last_peered":"2024-08-01T04:14:50.032101+0000","last_clean":"2024-08-01T04:14:50.032101+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:14:50.032101+0000","last_undegraded":"2024-08-01T04:14:50.032101+0000","last_fullsized":"2024-08-01T04:14:50.032101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035723+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:14:50.035723+0000","last_peered":"2024-08-01T04:14:50.035723+0000","last_clean":"2024-08-01T04:14:50.035723+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:14:50.035723+0000","last_undegraded":"2024-08-01T04:14:50.035723+0000","last_fullsized":"2024-08-01T04:14:50.035723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031481+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:14:50.031481+0000","last_peered":"2024-08-01T04:14:50.031481+0000","last_clean":"2024-08-01T04:14:50.031481+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:14:50.031481+0000","last_undegraded":"2024-08-01T04:14:50.031481+0000","last_fullsized":"2024-08-01T04:14:50.031481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033154+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:14:50.033154+0000","last_peered":"2024-08-01T04:14:50.033154+0000","last_clean":"2024-08-01T04:14:50.033154+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:14:50.033154+0000","last_undegraded":"2024-08-01T04:14:50.033154+0000","last_fullsized":"2024-08-01T04:14:50.033154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032021+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:14:50.032021+0000","last_peered":"2024-08-01T04:14:50.032021+0000","last_clean":"2024-08-01T04:14:50.032021+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:14:50.032021+0000","last_undegraded":"2024-08-01T04:14:50.032021+0000","last_fullsized":"2024-08-01T04:14:50.032021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031484+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:14:50.031484+0000","last_peered":"2024-08-01T04:14:50.031484+0000","last_clean":"2024-08-01T04:14:50.031484+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:14:50.031484+0000","last_undegraded":"2024-08-01T04:14:50.031484+0000","last_fullsized":"2024-08-01T04:14:50.031484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.039870+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:14:50.039870+0000","last_peered":"2024-08-01T04:14:50.039870+0000","last_clean":"2024-08-01T04:14:50.039870+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:14:50.039870+0000","last_undegraded":"2024-08-01T04:14:50.039870+0000","last_fullsized":"2024-08-01T04:14:50.039870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041066+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:14:50.041066+0000","last_peered":"2024-08-01T04:14:50.041066+0000","last_clean":"2024-08-01T04:14:50.041066+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:14:50.041066+0000","last_undegraded":"2024-08-01T04:14:50.041066+0000","last_fullsized":"2024-08-01T04:14:50.041066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031367+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:14:50.031367+0000","last_peered":"2024-08-01T04:14:50.031367+0000","last_clean":"2024-08-01T04:14:50.031367+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:14:50.031367+0000","last_undegraded":"2024-08-01T04:14:50.031367+0000","last_fullsized":"2024-08-01T04:14:50.031367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041790+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:14:50.041790+0000","last_peered":"2024-08-01T04:14:50.041790+0000","last_clean":"2024-08-01T04:14:50.041790+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:14:50.041790+0000","last_undegraded":"2024-08-01T04:14:50.041790+0000","last_fullsized":"2024-08-01T04:14:50.041790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:15:13.747169+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:15:13.747169+0000","last_peered":"2024-08-01T04:15:13.747169+0000","last_clean":"2024-08-01T04:15:13.747169+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:15:13.747169+0000","last_undegraded":"2024-08-01T04:15:13.747169+0000","last_fullsized":"2024-08-01T04:15:13.747169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.035183+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:14:50.035183+0000","last_peered":"2024-08-01T04:14:50.035183+0000","last_clean":"2024-08-01T04:14:50.035183+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:14:50.035183+0000","last_undegraded":"2024-08-01T04:14:50.035183+0000","last_fullsized":"2024-08-01T04:14:50.035183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031391+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:14:50.031391+0000","last_peered":"2024-08-01T04:14:50.031391+0000","last_clean":"2024-08-01T04:14:50.031391+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:14:50.031391+0000","last_undegraded":"2024-08-01T04:14:50.031391+0000","last_fullsized":"2024-08-01T04:14:50.031391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029513+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:14:50.029513+0000","last_peered":"2024-08-01T04:14:50.029513+0000","last_clean":"2024-08-01T04:14:50.029513+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:14:50.029513+0000","last_undegraded":"2024-08-01T04:14:50.029513+0000","last_fullsized":"2024-08-01T04:14:50.029513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032882+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:14:50.032882+0000","last_peered":"2024-08-01T04:14:50.032882+0000","last_clean":"2024-08-01T04:14:50.032882+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:14:50.032882+0000","last_undegraded":"2024-08-01T04:14:50.032882+0000","last_fullsized":"2024-08-01T04:14:50.032882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029710+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:14:50.029710+0000","last_peered":"2024-08-01T04:14:50.029710+0000","last_clean":"2024-08-01T04:14:50.029710+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:14:50.029710+0000","last_undegraded":"2024-08-01T04:14:50.029710+0000","last_fullsized":"2024-08-01T04:14:50.029710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032718+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:14:50.032718+0000","last_peered":"2024-08-01T04:14:50.032718+0000","last_clean":"2024-08-01T04:14:50.032718+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:14:50.032718+0000","last_undegraded":"2024-08-01T04:14:50.032718+0000","last_fullsized":"2024-08-01T04:14:50.032718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041070+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:14:50.041070+0000","last_peered":"2024-08-01T04:14:50.041070+0000","last_clean":"2024-08-01T04:14:50.041070+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:14:50.041070+0000","last_undegraded":"2024-08-01T04:14:50.041070+0000","last_fullsized":"2024-08-01T04:14:50.041070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032616+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:14:50.032616+0000","last_peered":"2024-08-01T04:14:50.032616+0000","last_clean":"2024-08-01T04:14:50.032616+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:14:50.032616+0000","last_undegraded":"2024-08-01T04:14:50.032616+0000","last_fullsized":"2024-08-01T04:14:50.032616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042981+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:14:50.042981+0000","last_peered":"2024-08-01T04:14:50.042981+0000","last_clean":"2024-08-01T04:14:50.042981+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:14:50.042981+0000","last_undegraded":"2024-08-01T04:14:50.042981+0000","last_fullsized":"2024-08-01T04:14:50.042981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.040400+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:14:50.040400+0000","last_peered":"2024-08-01T04:14:50.040400+0000","last_clean":"2024-08-01T04:14:50.040400+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:14:50.040400+0000","last_undegraded":"2024-08-01T04:14:50.040400+0000","last_fullsized":"2024-08-01T04:14:50.040400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.051995+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:14:50.051995+0000","last_peered":"2024-08-01T04:14:50.051995+0000","last_clean":"2024-08-01T04:14:50.051995+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:14:50.051995+0000","last_undegraded":"2024-08-01T04:14:50.051995+0000","last_fullsized":"2024-08-01T04:14:50.051995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032827+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:14:50.032827+0000","last_peered":"2024-08-01T04:14:50.032827+0000","last_clean":"2024-08-01T04:14:50.032827+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:14:50.032827+0000","last_undegraded":"2024-08-01T04:14:50.032827+0000","last_fullsized":"2024-08-01T04:14:50.032827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032073+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:14:50.032073+0000","last_peered":"2024-08-01T04:14:50.032073+0000","last_clean":"2024-08-01T04:14:50.032073+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:14:50.032073+0000","last_undegraded":"2024-08-01T04:14:50.032073+0000","last_fullsized":"2024-08-01T04:14:50.032073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.043063+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:14:50.043063+0000","last_peered":"2024-08-01T04:14:50.043063+0000","last_clean":"2024-08-01T04:14:50.043063+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:14:50.043063+0000","last_undegraded":"2024-08-01T04:14:50.043063+0000","last_fullsized":"2024-08-01T04:14:50.043063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.029561+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:14:50.029561+0000","last_peered":"2024-08-01T04:14:50.029561+0000","last_clean":"2024-08-01T04:14:50.029561+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:14:50.029561+0000","last_undegraded":"2024-08-01T04:14:50.029561+0000","last_fullsized":"2024-08-01T04:14:50.029561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.052459+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:14:50.052459+0000","last_peered":"2024-08-01T04:14:50.052459+0000","last_clean":"2024-08-01T04:14:50.052459+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:14:50.052459+0000","last_undegraded":"2024-08-01T04:14:50.052459+0000","last_fullsized":"2024-08-01T04:14:50.052459+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.038977+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:14:50.038977+0000","last_peered":"2024-08-01T04:14:50.038977+0000","last_clean":"2024-08-01T04:14:50.038977+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:14:50.038977+0000","last_undegraded":"2024-08-01T04:14:50.038977+0000","last_fullsized":"2024-08-01T04:14:50.038977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032324+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:14:50.032324+0000","last_peered":"2024-08-01T04:14:50.032324+0000","last_clean":"2024-08-01T04:14:50.032324+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:14:50.032324+0000","last_undegraded":"2024-08-01T04:14:50.032324+0000","last_fullsized":"2024-08-01T04:14:50.032324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.031199+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:14:50.031199+0000","last_peered":"2024-08-01T04:14:50.031199+0000","last_clean":"2024-08-01T04:14:50.031199+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:14:50.031199+0000","last_undegraded":"2024-08-01T04:14:50.031199+0000","last_fullsized":"2024-08-01T04:14:50.031199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.037426+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:14:50.037426+0000","last_peered":"2024-08-01T04:14:50.037426+0000","last_clean":"2024-08-01T04:14:50.037426+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:14:50.037426+0000","last_undegraded":"2024-08-01T04:14:50.037426+0000","last_fullsized":"2024-08-01T04:14:50.037426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032792+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:14:50.032792+0000","last_peered":"2024-08-01T04:14:50.032792+0000","last_clean":"2024-08-01T04:14:50.032792+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:14:50.032792+0000","last_undegraded":"2024-08-01T04:14:50.032792+0000","last_fullsized":"2024-08-01T04:14:50.032792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032849+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:14:50.032849+0000","last_peered":"2024-08-01T04:14:50.032849+0000","last_clean":"2024-08-01T04:14:50.032849+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:14:50.032849+0000","last_undegraded":"2024-08-01T04:14:50.032849+0000","last_fullsized":"2024-08-01T04:14:50.032849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.032537+0000","last_change":"2024-08-01T04:13:51.461259+0000","last_active":"2024-08-01T04:14:50.032537+0000","last_peered":"2024-08-01T04:14:50.032537+0000","last_clean":"2024-08-01T04:14:50.032537+0000","last_became_active":"2024-08-01T04:13:51.460884+0000","last_became_peered":"2024-08-01T04:13:51.460884+0000","last_unstale":"2024-08-01T04:14:50.032537+0000","last_undegraded":"2024-08-01T04:14:50.032537+0000","last_fullsized":"2024-08-01T04:14:50.032537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:18.454778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.046684+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:14:50.046684+0000","last_peered":"2024-08-01T04:14:50.046684+0000","last_clean":"2024-08-01T04:14:50.046684+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:14:50.046684+0000","last_undegraded":"2024-08-01T04:14:50.046684+0000","last_fullsized":"2024-08-01T04:14:50.046684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.042436+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:14:50.042436+0000","last_peered":"2024-08-01T04:14:50.042436+0000","last_clean":"2024-08-01T04:14:50.042436+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:14:50.042436+0000","last_undegraded":"2024-08-01T04:14:50.042436+0000","last_fullsized":"2024-08-01T04:14:50.042436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.041784+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:14:50.041784+0000","last_peered":"2024-08-01T04:14:50.041784+0000","last_clean":"2024-08-01T04:14:50.041784+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:14:50.041784+0000","last_undegraded":"2024-08-01T04:14:50.041784+0000","last_fullsized":"2024-08-01T04:14:50.041784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-01T04:14:50.033142+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:14:50.033142+0000","last_peered":"2024-08-01T04:14:50.033142+0000","last_clean":"2024-08-01T04:14:50.033142+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:14:50.033142+0000","last_undegraded":"2024-08-01T04:14:50.033142+0000","last_fullsized":"2024-08-01T04:14:50.033142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66724,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":432,"num_read_kb":286,"num_write":166,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":211962,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":467596,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1121801,"data_compressed":41648,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":467832,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":467752,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1060071,"data_compressed":41648,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":467221,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":1059365,"data_compressed":41648,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":468048,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":467532,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531088,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":529348,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":468492,"data_compressed":5774,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34233,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:15:20.765 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-01T04:15:20.766 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-01T04:15:20.766 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-01T04:15:20.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph health --format=json 2024-08-01T04:15:21.295 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:21.467 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:21 smithi003 ceph-mon[32262]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-01T04:15:21.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:21 smithi163 ceph-mon[37383]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-01T04:15:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:21 smithi160 ceph-mon[37224]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-01T04:15:21.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 -- 172.21.15.3:0/3303922412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8488076d70 msgr2=0x7f8488112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:21.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 --2- 172.21.15.3:0/3303922412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8488076d70 0x7f8488112080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8470009900 tx=0x7f847002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:15:21.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 -- 172.21.15.3:0/3303922412 shutdown_connections 2024-08-01T04:15:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 --2- 172.21.15.3:0/3303922412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84881125c0 0x7f84881149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 --2- 172.21.15.3:0/3303922412 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8488076d70 0x7f8488112080 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 --2- 172.21.15.3:0/3303922412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8488076450 0x7f8488076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 -- 172.21.15.3:0/3303922412 >> 172.21.15.3:0/3303922412 conn(0x7f84880ff5e0 msgr2=0x7f8488101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 -- 172.21.15.3:0/3303922412 shutdown_connections 2024-08-01T04:15:21.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.922+0000 7f848d400640 1 -- 172.21.15.3:0/3303922412 wait complete. 2024-08-01T04:15:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.923+0000 7f848d400640 1 Processor -- start 2024-08-01T04:15:21.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.923+0000 7f848d400640 1 -- start start 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8488076450 0x7f848819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8488076d70 0x7f848819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 0x7f84881a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848d400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f84881182a0 con 0x7f84881125c0 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848d400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8488118120 con 0x7f8488076d70 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848d400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8488118420 con 0x7f8488076450 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 0x7f84881a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 0x7f84881a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50122/0 (socket says 172.21.15.3:50122) 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848ca00640 1 -- 172.21.15.3:0/103458903 learned_addr learned my addr 172.21.15.3:0/103458903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f8487e00640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8488076450 0x7f848819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:21.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f847f400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8488076d70 0x7f848819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848ca00640 1 -- 172.21.15.3:0/103458903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8488076450 msgr2=0x7f848819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848ca00640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8488076450 0x7f848819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.924+0000 7f848ca00640 1 -- 172.21.15.3:0/103458903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8488076d70 msgr2=0x7f848819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f848ca00640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8488076d70 0x7f848819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f848ca00640 1 -- 172.21.15.3:0/103458903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84881a36d0 con 0x7f84881125c0 2024-08-01T04:15:21.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f848ca00640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 0x7f84881a2ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f848000eb10 tx=0x7f848000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:21.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480004b20 con 0x7f84881125c0 2024-08-01T04:15:21.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8480004cc0 con 0x7f84881125c0 2024-08-01T04:15:21.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84881a39c0 con 0x7f84881125c0 2024-08-01T04:15:21.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480010790 con 0x7f84881125c0 2024-08-01T04:15:21.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.925+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84881ab2a0 con 0x7f84881125c0 2024-08-01T04:15:21.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.927+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8488077b90 con 0x7f84881125c0 2024-08-01T04:15:21.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.928+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8480005020 con 0x7f84881125c0 2024-08-01T04:15:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.930+0000 7f8485600640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f846c0a2c90 0x7f846c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.930+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f84800cf8c0 con 0x7f84881125c0 2024-08-01T04:15:21.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.930+0000 7f8487e00640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f846c0a2c90 0x7f846c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:21.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.931+0000 7f8487e00640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f846c0a2c90 0x7f846c0a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f847800bbe0 tx=0x7f847800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:21.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:21.935+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f848009c640 con 0x7f84881125c0 2024-08-01T04:15:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.171+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f84881ab470 con 0x7f84881125c0 2024-08-01T04:15:22.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.172+0000 7f8485600640 1 -- 172.21.15.3:0/103458903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f848009bf40 con 0x7f84881125c0 2024-08-01T04:15:22.174 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:22.174 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-01T04:15:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f846c0a2c90 msgr2=0x7f846c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f846c0a2c90 0x7f846c0a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f847800bbe0 tx=0x7f847800b390 comp rx=0 tx=0).stop 2024-08-01T04:15:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 msgr2=0x7f84881a2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 0x7f84881a2ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f848000eb10 tx=0x7f848000c2d0 comp rx=0 tx=0).stop 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 shutdown_connections 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f846c0a2c90 0x7f846c0a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84881125c0 0x7f84881a2ff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8488076d70 0x7f848819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 --2- 172.21.15.3:0/103458903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8488076450 0x7f848819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 >> 172.21.15.3:0/103458903 conn(0x7f84880ff5e0 msgr2=0x7f84881019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 shutdown_connections 2024-08-01T04:15:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:22.175+0000 7f848d400640 1 -- 172.21.15.3:0/103458903 wait complete. 2024-08-01T04:15:22.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:22 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/103458903' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-01T04:15:22.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:22 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/103458903' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-01T04:15:22.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:22 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/103458903' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-01T04:15:23.072 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-01T04:15:23.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:15:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:23 smithi003 ceph-mon[32262]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:23.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:23 smithi163 ceph-mon[37383]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:23.558 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:23.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:23 smithi160 ceph-mon[37224]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 -- 172.21.15.3:0/4033920782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b4103900 msgr2=0x7fc7b4103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 --2- 172.21.15.3:0/4033920782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b4103900 0x7fc7b4103d60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009930 tx=0x7fc79c02a420 comp rx=0 tx=0).stop 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 -- 172.21.15.3:0/4033920782 shutdown_connections 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 --2- 172.21.15.3:0/4033920782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b41042a0 0x7fc7b41107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 --2- 172.21.15.3:0/4033920782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b4103900 0x7fc7b4103d60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 --2- 172.21.15.3:0/4033920782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7b4109900 0x7fc7b4109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.226+0000 7fc7b3200640 1 -- 172.21.15.3:0/4033920782 >> 172.21.15.3:0/4033920782 conn(0x7fc7b40ff600 msgr2=0x7fc7b4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.227+0000 7fc7b3200640 1 -- 172.21.15.3:0/4033920782 shutdown_connections 2024-08-01T04:15:24.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.227+0000 7fc7b3200640 1 -- 172.21.15.3:0/4033920782 wait complete. 2024-08-01T04:15:24.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 Processor -- start 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 -- start start 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 0x7fc7b419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b41042a0 0x7fc7b419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7b4109900 0x7fc7b419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4113d50 con 0x7fc7b41042a0 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4113bd0 con 0x7fc7b4103900 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.228+0000 7fc7b3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4113ed0 con 0x7fc7b4109900 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 0x7fc7b419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b41042a0 0x7fc7b419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:24.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 0x7fc7b419a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39422/0 (socket says 172.21.15.3:39422) 2024-08-01T04:15:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 -- 172.21.15.3:0/788933117 learned_addr learned my addr 172.21.15.3:0/788933117 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b2800640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7b4109900 0x7fc7b419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 -- 172.21.15.3:0/788933117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7b4109900 msgr2=0x7fc7b419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:24.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7b4109900 0x7fc7b419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 -- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b41042a0 msgr2=0x7fc7b419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b41042a0 0x7fc7b419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 -- 172.21.15.3:0/788933117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b419f1c0 con 0x7fc7b4103900 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.229+0000 7fc7b1e00640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 0x7fc7b419a210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400e9f0 tx=0x7fc7a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.230+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400cd80 con 0x7fc7b4103900 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.230+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b419f4b0 con 0x7fc7b4103900 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.230+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4005500 con 0x7fc7b4103900 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.230+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4010690 con 0x7fc7b4103900 2024-08-01T04:15:24.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.230+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a6d90 con 0x7fc7b4103900 2024-08-01T04:15:24.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.231+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b4103d60 con 0x7fc7b4103900 2024-08-01T04:15:24.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.233+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc7a40046c0 con 0x7fc7b4103900 2024-08-01T04:15:24.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.235+0000 7fc7aaa00640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc7980a2c90 0x7fc7980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.235+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fc7a40cf590 con 0x7fc7b4103900 2024-08-01T04:15:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.235+0000 7fc7abe00640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc7980a2c90 0x7fc7980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:24.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.236+0000 7fc7abe00640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc7980a2c90 0x7fc7980a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006010 tx=0x7fc79c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:24.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.237+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc7a409c310 con 0x7fc7b4103900 2024-08-01T04:15:24.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.469+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7b410bed0 con 0x7fc7b4103900 2024-08-01T04:15:24.472 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:24.472 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-08-01T04:14:12:627814+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:14:11.859614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:15:24.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.470+0000 7fc7aaa00640 1 -- 172.21.15.3:0/788933117 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7fc7a409bc10 con 0x7fc7b4103900 2024-08-01T04:15:24.473 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:15:24.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc7980a2c90 msgr2=0x7fc7980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:24.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc7980a2c90 0x7fc7980a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006010 tx=0x7fc79c000bc0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 msgr2=0x7fc7b419a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:24.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 0x7fc7b419a210 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400e9f0 tx=0x7fc7a400edf0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 shutdown_connections 2024-08-01T04:15:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc7980a2c90 0x7fc7980a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7b4109900 0x7fc7b419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7b41042a0 0x7fc7b419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 --2- 172.21.15.3:0/788933117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7b4103900 0x7fc7b419a210 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.473+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 >> 172.21.15.3:0/788933117 conn(0x7fc7b40ff600 msgr2=0x7fc7b40ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.474+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 shutdown_connections 2024-08-01T04:15:24.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:24.474+0000 7fc7b3200640 1 -- 172.21.15.3:0/788933117 wait complete. 2024-08-01T04:15:25.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:15:25.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:25 smithi163 ceph-mon[37383]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-01T04:15:25.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/788933117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:25.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:25 smithi003 ceph-mon[32262]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-01T04:15:25.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/788933117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:25.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:25 smithi160 ceph-mon[37224]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-01T04:15:25.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/788933117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:25.892 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:26.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.594+0000 7f43a0600640 1 -- 172.21.15.3:0/2317063003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 msgr2=0x7f4398109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:26.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.594+0000 7f43a0600640 1 --2- 172.21.15.3:0/2317063003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 0x7f4398109f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f438c009930 tx=0x7f438c02a440 comp rx=0 tx=0).stop 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 -- 172.21.15.3:0/2317063003 shutdown_connections 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 --2- 172.21.15.3:0/2317063003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398104470 0x7f43981109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 --2- 172.21.15.3:0/2317063003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398103ad0 0x7f4398103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 --2- 172.21.15.3:0/2317063003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 0x7f4398109f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 -- 172.21.15.3:0/2317063003 >> 172.21.15.3:0/2317063003 conn(0x7f43980ff750 msgr2=0x7f4398101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 -- 172.21.15.3:0/2317063003 shutdown_connections 2024-08-01T04:15:26.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.595+0000 7f43a0600640 1 -- 172.21.15.3:0/2317063003 wait complete. 2024-08-01T04:15:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.596+0000 7f43a0600640 1 Processor -- start 2024-08-01T04:15:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 -- start start 2024-08-01T04:15:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398103ad0 0x7f439819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104470 0x7f439819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:26.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 0x7f439819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4398113ee0 con 0x7f4398109b60 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4398113d60 con 0x7f4398104470 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f43a0600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4398114060 con 0x7f4398103ad0 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f439dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398103ad0 0x7f439819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f439d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104470 0x7f439819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f439dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398103ad0 0x7f439819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50018/0 (socket says 172.21.15.3:50018) 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f439dc00640 1 -- 172.21.15.3:0/3471080254 learned_addr learned my addr 172.21.15.3:0/3471080254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.597+0000 7f439e600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 0x7f439819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.598+0000 7f439d200640 1 -- 172.21.15.3:0/3471080254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398103ad0 msgr2=0x7f439819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.598+0000 7f439d200640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398103ad0 0x7f439819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.598+0000 7f439d200640 1 -- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 msgr2=0x7f439819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.598+0000 7f439d200640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 0x7f439819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.598+0000 7f439d200640 1 -- 172.21.15.3:0/3471080254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439819f2f0 con 0x7f4398104470 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.598+0000 7f439d200640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104470 0x7f439819a880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f438400c970 tx=0x7f438400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.599+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384007bb0 con 0x7f4398104470 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.599+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4384007d50 con 0x7f4398104470 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.599+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384018820 con 0x7f4398104470 2024-08-01T04:15:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.599+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439819f5e0 con 0x7f4398104470 2024-08-01T04:15:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.599+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981a6ec0 con 0x7f4398104470 2024-08-01T04:15:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.600+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4398103f30 con 0x7f4398104470 2024-08-01T04:15:26.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.603+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4384043b10 con 0x7f4398104470 2024-08-01T04:15:26.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.605+0000 7f4396000640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f43800a2c90 0x7f43800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.605+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f43840cf310 con 0x7f4398104470 2024-08-01T04:15:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.605+0000 7f439dc00640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f43800a2c90 0x7f43800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.606+0000 7f439dc00640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f43800a2c90 0x7f43800a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f438c0096c0 tx=0x7f438c005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.606+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4384014030 con 0x7f4398104470 2024-08-01T04:15:26.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.842+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43981a71e0 con 0x7f4398104470 2024-08-01T04:15:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.843+0000 7f4396000640 1 -- 172.21.15.3:0/3471080254 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f438409c110 con 0x7f4398104470 2024-08-01T04:15:26.845 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:26.845 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-08-01T04:14:12:627814+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:14:11.859614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:15:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:15:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f43800a2c90 msgr2=0x7f43800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f43800a2c90 0x7f43800a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f438c0096c0 tx=0x7f438c005b50 comp rx=0 tx=0).stop 2024-08-01T04:15:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104470 msgr2=0x7f439819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104470 0x7f439819a880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f438400c970 tx=0x7f438400ce30 comp rx=0 tx=0).stop 2024-08-01T04:15:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 shutdown_connections 2024-08-01T04:15:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f43800a2c90 0x7f43800a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4398109b60 0x7f439819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104470 0x7f439819a880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 --2- 172.21.15.3:0/3471080254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4398103ad0 0x7f439819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 >> 172.21.15.3:0/3471080254 conn(0x7f43980ff750 msgr2=0x7f43980fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 shutdown_connections 2024-08-01T04:15:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:26.847+0000 7f43a0600640 1 -- 172.21.15.3:0/3471080254 wait complete. 2024-08-01T04:15:27.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:27 smithi163 ceph-mon[37383]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-01T04:15:27.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:27 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3471080254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:27 smithi003 ceph-mon[32262]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-01T04:15:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:27 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3471080254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:27 smithi160 ceph-mon[37224]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-08-01T04:15:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:27 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3471080254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:27.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:15:28.162 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:28.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.824+0000 7fa556000640 1 -- 172.21.15.3:0/3562882605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 msgr2=0x7fa550103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:28.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.824+0000 7fa556000640 1 --2- 172.21.15.3:0/3562882605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 0x7fa550103f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009860 tx=0x7fa53c02a340 comp rx=0 tx=0).stop 2024-08-01T04:15:28.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 -- 172.21.15.3:0/3562882605 shutdown_connections 2024-08-01T04:15:28.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 --2- 172.21.15.3:0/3562882605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa550104460 0x7fa550110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:28.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 --2- 172.21.15.3:0/3562882605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 0x7fa550103f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:28.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 --2- 172.21.15.3:0/3562882605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550109ac0 0x7fa550109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:28.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 -- 172.21.15.3:0/3562882605 >> 172.21.15.3:0/3562882605 conn(0x7fa5500ff7c0 msgr2=0x7fa550101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:28.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 -- 172.21.15.3:0/3562882605 shutdown_connections 2024-08-01T04:15:28.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.825+0000 7fa556000640 1 -- 172.21.15.3:0/3562882605 wait complete. 2024-08-01T04:15:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.826+0000 7fa556000640 1 Processor -- start 2024-08-01T04:15:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.826+0000 7fa556000640 1 -- start start 2024-08-01T04:15:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa556000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 0x7fa55019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa556000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 0x7fa55019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa556000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa550109ac0 0x7fa55019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:28.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa556000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa550113ec0 con 0x7fa550109ac0 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa556000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa550113d40 con 0x7fa550103ac0 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa556000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa550114040 con 0x7fa550104460 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa54ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 0x7fa55019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa54f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 0x7fa55019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa54ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 0x7fa55019a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50036/0 (socket says 172.21.15.3:50036) 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa54ea00640 1 -- 172.21.15.3:0/3016929025 learned_addr learned my addr 172.21.15.3:0/3016929025 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:28.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.827+0000 7fa54fe00640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa550109ac0 0x7fa55019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa54ea00640 1 -- 172.21.15.3:0/3016929025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 msgr2=0x7fa55019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa54ea00640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 0x7fa55019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:28.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa54ea00640 1 -- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa550109ac0 msgr2=0x7fa55019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa54ea00640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa550109ac0 0x7fa55019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa54ea00640 1 -- 172.21.15.3:0/3016929025 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55019f2d0 con 0x7fa550104460 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa54ea00640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 0x7fa55019a860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa53c035040 tx=0x7fa53c033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c037040 con 0x7fa550104460 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c040070 con 0x7fa550104460 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c003a10 con 0x7fa550104460 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.828+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55019f560 con 0x7fa550104460 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.829+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5501a6ea0 con 0x7fa550104460 2024-08-01T04:15:28.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.830+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa550103f20 con 0x7fa550104460 2024-08-01T04:15:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.834+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa53c004150 con 0x7fa550104460 2024-08-01T04:15:28.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.835+0000 7fa547e00640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa5340a2c10 0x7fa5340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:28.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.835+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fa53c0ef080 con 0x7fa550104460 2024-08-01T04:15:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.835+0000 7fa54f400640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa5340a2c10 0x7fa5340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.836+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa53c049020 con 0x7fa550104460 2024-08-01T04:15:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:28.836+0000 7fa54f400640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa5340a2c10 0x7fa5340a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5400096c0 tx=0x7fa540009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:29.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.035+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa55019b680 con 0x7fa550104460 2024-08-01T04:15:29.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.036+0000 7fa547e00640 1 -- 172.21.15.3:0/3016929025 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22657 (secure 0 0 0) 0x7fa55019b680 con 0x7fa550104460 2024-08-01T04:15:29.038 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":88,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:14:50.023517+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.039+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa5340a2c10 msgr2=0x7fa5340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.039+0000 7fa556000640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa5340a2c10 0x7fa5340a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5400096c0 tx=0x7fa540009260 comp rx=0 tx=0).stop 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 msgr2=0x7fa55019a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 0x7fa55019a860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa53c035040 tx=0x7fa53c033b60 comp rx=0 tx=0).stop 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 shutdown_connections 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fa5340a2c10 0x7fa5340a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:29.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa550109ac0 0x7fa55019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa550104460 0x7fa55019a860 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 --2- 172.21.15.3:0/3016929025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa550103ac0 0x7fa55019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 >> 172.21.15.3:0/3016929025 conn(0x7fa5500ff7c0 msgr2=0x7fa5500fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 shutdown_connections 2024-08-01T04:15:29.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:29.040+0000 7fa556000640 1 -- 172.21.15.3:0/3016929025 wait complete. 2024-08-01T04:15:29.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:29 smithi003 ceph-mon[32262]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:29.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:29 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3016929025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:15:29.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:29 smithi163 ceph-mon[37383]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:29.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:29 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3016929025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:15:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:29 smithi160 ceph-mon[37224]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:29 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3016929025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:15:30.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:15:30.508 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 -- 172.21.15.3:0/2165003598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9640107d50 msgr2=0x7f9640108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 --2- 172.21.15.3:0/2165003598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9640107d50 0x7f9640108170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9628009930 tx=0x7f962802a420 comp rx=0 tx=0).stop 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 -- 172.21.15.3:0/2165003598 shutdown_connections 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 --2- 172.21.15.3:0/2165003598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96401086b0 0x7f9640114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 --2- 172.21.15.3:0/2165003598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9640107d50 0x7f9640108170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 --2- 172.21.15.3:0/2165003598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f964010dd90 0x7f964010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 -- 172.21.15.3:0/2165003598 >> 172.21.15.3:0/2165003598 conn(0x7f96400776b0 msgr2=0x7f9640079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:31.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 -- 172.21.15.3:0/2165003598 shutdown_connections 2024-08-01T04:15:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.181+0000 7f963f200640 1 -- 172.21.15.3:0/2165003598 wait complete. 2024-08-01T04:15:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.182+0000 7f963f200640 1 Processor -- start 2024-08-01T04:15:31.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.182+0000 7f963f200640 1 -- start start 2024-08-01T04:15:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9640107d50 0x7f964019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96401086b0 0x7f964019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f964010dd90 0x7f96401a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f96401180b0 con 0x7f96401086b0 2024-08-01T04:15:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9640117f30 con 0x7f964010dd90 2024-08-01T04:15:31.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9640118230 con 0x7f9640107d50 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f9637e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96401086b0 0x7f964019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9640107d50 0x7f964019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f9637e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96401086b0 0x7f964019eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50208/0 (socket says 172.21.15.3:50208) 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f9637e00640 1 -- 172.21.15.3:0/3573719836 learned_addr learned my addr 172.21.15.3:0/3573719836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963e800640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f964010dd90 0x7f96401a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963de00640 1 -- 172.21.15.3:0/3573719836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f964010dd90 msgr2=0x7f96401a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.183+0000 7f963de00640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f964010dd90 0x7f96401a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f963de00640 1 -- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96401086b0 msgr2=0x7f964019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f963de00640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96401086b0 0x7f964019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f963de00640 1 -- 172.21.15.3:0/3573719836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401a35a0 con 0x7f9640107d50 2024-08-01T04:15:31.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f963de00640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9640107d50 0x7f964019e5f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f963000e9f0 tx=0x7f963000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963000ccc0 con 0x7f9640107d50 2024-08-01T04:15:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96401a3890 con 0x7f9640107d50 2024-08-01T04:15:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630005020 con 0x7f9640107d50 2024-08-01T04:15:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300049e0 con 0x7f9640107d50 2024-08-01T04:15:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.184+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401ab170 con 0x7f9640107d50 2024-08-01T04:15:31.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.185+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9640108170 con 0x7f9640107d50 2024-08-01T04:15:31.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.187+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9630005560 con 0x7f9640107d50 2024-08-01T04:15:31.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.189+0000 7f9636a00640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f96240a2c90 0x7f96240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:31.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.189+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f96300cf150 con 0x7f9640107d50 2024-08-01T04:15:31.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.189+0000 7f9637e00640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f96240a2c90 0x7f96240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:31.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.190+0000 7f9637e00640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f96240a2c90 0x7f96240a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f96401078c0 tx=0x7f9628009560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:31.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.193+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f96300051d0 con 0x7f9640107d50 2024-08-01T04:15:31.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.433+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9640110280 con 0x7f9640107d50 2024-08-01T04:15:31.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.434+0000 7f9636a00640 1 -- 172.21.15.3:0/3573719836 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f963009bed0 con 0x7f9640107d50 2024-08-01T04:15:31.435 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:31.436 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-08-01T04:14:12:627814+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:14:11.859614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:15:31.436 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:15:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f96240a2c90 msgr2=0x7f96240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:31.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f96240a2c90 0x7f96240a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f96401078c0 tx=0x7f9628009560 comp rx=0 tx=0).stop 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9640107d50 msgr2=0x7f964019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9640107d50 0x7f964019e5f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f963000e9f0 tx=0x7f963000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 shutdown_connections 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f96240a2c90 0x7f96240a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f964010dd90 0x7f96401a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96401086b0 0x7f964019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 --2- 172.21.15.3:0/3573719836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9640107d50 0x7f964019e5f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.437+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 >> 172.21.15.3:0/3573719836 conn(0x7f96400776b0 msgr2=0x7f9640078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.438+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 shutdown_connections 2024-08-01T04:15:31.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:31.438+0000 7f963f200640 1 -- 172.21.15.3:0/3573719836 wait complete. 2024-08-01T04:15:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:31 smithi163 ceph-mon[37383]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:31.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:31 smithi003 ceph-mon[32262]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:31.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:31 smithi160 ceph-mon[37224]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:32.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-01T04:13:55.770404+0000', 'modified': '2024-08-01T04:14:11.859614+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24520, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/3179303046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 3179303046}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 3179303046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/1698078935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1698078935}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1698078935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6839/4057809310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 4057809310}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 4057809310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.160:6837/3127569351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 3127569351}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 3127569351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.3:6841/2435488646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 2435488646}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 2435488646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.160:6839/1741438883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 1741438883}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 1741438883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24548, 'qdb_cluster': [24548, 24520, 24506]} 2024-08-01T04:15:32.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-01T04:15:32.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.h status 2024-08-01T04:15:32.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:32 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3573719836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:32.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:32 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3573719836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:32.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:32 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3573719836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:32.808 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:33 smithi003 ceph-mon[32262]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 -- 172.21.15.3:0/751434764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 msgr2=0x7fab9c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 --2- 172.21.15.3:0/751434764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c108170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fab84009930 tx=0x7fab8402a440 comp rx=0 tx=0).stop 2024-08-01T04:15:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 -- 172.21.15.3:0/751434764 shutdown_connections 2024-08-01T04:15:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 --2- 172.21.15.3:0/751434764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab9c1086b0 0x7fab9c114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 --2- 172.21.15.3:0/751434764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c108170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 --2- 172.21.15.3:0/751434764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c10dd90 0x7fab9c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 -- 172.21.15.3:0/751434764 >> 172.21.15.3:0/751434764 conn(0x7fab9c0776b0 msgr2=0x7fab9c079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 -- 172.21.15.3:0/751434764 shutdown_connections 2024-08-01T04:15:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.573+0000 7faba1600640 1 -- 172.21.15.3:0/751434764 wait complete. 2024-08-01T04:15:33.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.574+0000 7faba1600640 1 Processor -- start 2024-08-01T04:15:33.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.574+0000 7faba1600640 1 -- start start 2024-08-01T04:15:33.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba1600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c1086b0 0x7fab9c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba1600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab9c10dd90 0x7fab9c1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba1600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c1180e0 con 0x7fab9c107d50 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba1600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c117f60 con 0x7fab9c1086b0 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba1600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c118260 con 0x7fab9c10dd90 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7fab9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7fab93e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c1086b0 0x7fab9c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7fab9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51268/0 (socket says 172.21.15.3:51268) 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7fab93e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c1086b0 0x7fab9c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34546/0 (socket says 172.21.15.3:34546) 2024-08-01T04:15:33.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7fab9be00640 1 -- 172.21.15.3:0/3064752999 learned_addr learned my addr 172.21.15.3:0/3064752999 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:33.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.575+0000 7faba0c00640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab9c10dd90 0x7fab9c1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab9be00640 1 -- 172.21.15.3:0/3064752999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab9c10dd90 msgr2=0x7fab9c1a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab9be00640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab9c10dd90 0x7fab9c1a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab9be00640 1 -- 172.21.15.3:0/3064752999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c1086b0 msgr2=0x7fab9c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab9be00640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c1086b0 0x7fab9c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab9be00640 1 -- 172.21.15.3:0/3064752999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c1a35b0 con 0x7fab9c107d50 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab9be00640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c19e600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00ea60 tx=0x7fab8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab99600640 1 -- 172.21.15.3:0/3064752999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00cde0 con 0x7fab9c107d50 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab99600640 1 -- 172.21.15.3:0/3064752999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c0035f0 con 0x7fab9c107d50 2024-08-01T04:15:33.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7fab99600640 1 -- 172.21.15.3:0/3064752999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c010670 con 0x7fab9c107d50 2024-08-01T04:15:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.576+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c1a38a0 con 0x7fab9c107d50 2024-08-01T04:15:33.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.577+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c1ab180 con 0x7fab9c107d50 2024-08-01T04:15:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.580+0000 7faba57e2740 1 Processor -- start 2024-08-01T04:15:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.580+0000 7fab99600640 1 -- 172.21.15.3:0/3064752999 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fab8c0037a0 con 0x7fab9c107d50 2024-08-01T04:15:33.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.580+0000 7faba57e2740 1 -- start start 2024-08-01T04:15:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba57e2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x563969887fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba57e2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x563969888920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba57e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969893280 0x563969895670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba57e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5639697ee940 con 0x563969893280 2024-08-01T04:15:33.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba57e2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5639697869a0 con 0x563969889a50 2024-08-01T04:15:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba57e2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5639697ea900 con 0x5639698884e0 2024-08-01T04:15:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7fab93e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969893280 0x563969895670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7fab9be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x563969888920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7faba0c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x563969887fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7fab9be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x563969888920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2024-08-01T04:15:33.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.581+0000 7fab9be00640 1 -- 172.21.15.3:0/2874029005 learned_addr learned my addr 172.21.15.3:0/2874029005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:33.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab9be00640 1 -- 172.21.15.3:0/2874029005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 msgr2=0x563969887fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab9be00640 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x563969887fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab9be00640 1 -- 172.21.15.3:0/2874029005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969893280 msgr2=0x563969895670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab9be00640 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969893280 0x563969895670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab9be00640 1 -- 172.21.15.3:0/2874029005 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563969774ef0 con 0x5639698884e0 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7faba0c00640 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x563969887fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab9be00640 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x563969888920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0d73f0 tx=0x7fab8c0d78b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0bc22a7fc43afb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.582+0000 7fab99600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fab800a2c90 0x7fab800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.583+0000 7fab91200640 1 -- 172.21.15.3:0/2874029005 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0e3070 con 0x5639698884e0 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.583+0000 7faba0c00640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fab800a2c90 0x7fab800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.583+0000 7fab99600640 1 -- 172.21.15.3:0/3064752999 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fab8c0cf420 con 0x7fab9c107d50 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.583+0000 7fab91200640 1 -- 172.21.15.3:0/2874029005 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c09c1a0 con 0x5639698884e0 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.583+0000 7fab91200640 1 -- 172.21.15.3:0/2874029005 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c09b5e0 con 0x5639698884e0 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 -- 172.21.15.3:0/2874029005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 msgr2=0x563969888920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x563969888920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0d73f0 tx=0x7fab8c0d78b0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba0c00640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fab800a2c90 0x7fab800a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab94005fd0 tx=0x7fab94009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 -- 172.21.15.3:0/2874029005 shutdown_connections 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969893280 0x563969895670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x563969888920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 --2- 172.21.15.3:0/2874029005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x563969887fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 -- 172.21.15.3:0/2874029005 >> 172.21.15.3:0/2874029005 conn(0x563969850a00 msgr2=0x563969884b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 -- 172.21.15.3:0/2874029005 shutdown_connections 2024-08-01T04:15:33.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 -- 172.21.15.3:0/2874029005 wait complete. 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.584+0000 7faba57e2740 1 -- start start 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 Processor -- start 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x563969890290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x5639698907d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x563969890d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563969895f80 con 0x5639698884e0 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5639697a9a80 con 0x563969889a50 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563969896100 con 0x563969893280 2024-08-01T04:15:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.585+0000 7faba57e2740 10 client.0 _ll_register_callbacks cb 0x563969772140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x563969890290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x5639698907d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x563969890290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51282/0 (socket says 172.21.15.3:51282) 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab9be00640 1 -- 172.21.15.3:0/67986135 learned_addr learned my addr 172.21.15.3:0/67986135 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7faba0c00640 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x563969890d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 -- 172.21.15.3:0/67986135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 msgr2=0x563969890d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x563969890d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 -- 172.21.15.3:0/67986135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 msgr2=0x563969890290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x563969890290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 -- 172.21.15.3:0/67986135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396978d700 con 0x563969889a50 2024-08-01T04:15:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab93e00640 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x5639698907d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab84009230 tx=0x7fab840043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24c2ab083e1383 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.586+0000 7fab89600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7fab89600640 1 -- 172.21.15.3:0/67986135 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84035040 con 0x563969889a50 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8c09c1a0 con 0x563969889a50 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7fab89600640 1 -- 172.21.15.3:0/67986135 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fab84005450 con 0x563969889a50 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7fab89600640 1 -- 172.21.15.3:0/67986135 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab8404c070 con 0x563969889a50 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7faba57e2740 10 client.24890 fetch_fsmap learned FSMap version 25 2024-08-01T04:15:33.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639695602b0 con 0x563969889a50 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7fab89600640 1 -- 172.21.15.3:0/67986135 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab8c09c1a0 con 0x563969889a50 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.587+0000 7fab89600640 1 -- 172.21.15.3:0/67986135 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab84031850 con 0x563969889a50 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7faba57e2740 10 client.24890 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7faba57e2740 10 client.24890 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7faba57e2740 20 client.24890 populate_metadata read hostname 'smithi003' 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7faba57e2740 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x5639698a4130 0x5639698a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7faba57e2740 4 client.24890 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639697fbeb0 con 0x5639698a4130 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.588+0000 7fab9be00640 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x5639698a4130 0x5639698a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.589+0000 7fab9be00640 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x5639698a4130 0x5639698a55a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e048f24f29183008 server_cookie=78e72beb3e4df8d6 in_seq=0 out_seq=0 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.589+0000 7fab89600640 10 client.24890 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:15:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7fab89600640 1 -- 172.21.15.3:0/67986135 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5639697fbeb0 con 0x5639698a4130 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7fab89600640 10 client.24890 handle_command_reply: tid=0 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 client.24890 shutdown 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 msgr2=0x5639698907d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x5639698907d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fab84009230 tx=0x7fab840043b0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 shutdown_connections 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x5639698a4130 0x5639698a55a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x563969890d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x5639698907d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 --2- 172.21.15.3:0/67986135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x563969890290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7fab89600640 0 client.24890 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 >> 172.21.15.3:0/67986135 conn(0x563969850a00 msgr2=0x56396988a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 shutdown_connections 2024-08-01T04:15:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 1 -- 172.21.15.3:0/67986135 wait complete. 2024-08-01T04:15:33.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.590+0000 7faba57e2740 20 client.24890 trim_cache size 0 max 16384 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.610+0000 7faba57e2740 1 Processor -- start 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.610+0000 7faba57e2740 1 -- start start 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba57e2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x5639698a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba57e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969889a50 0x563969884d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba57e2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 0x563969890b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba57e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5639698a38e0 con 0x563969889a50 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba57e2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5639698a3fb0 con 0x563969893280 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba57e2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5639698a4680 con 0x5639698884e0 2024-08-01T04:15:33.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba0c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 0x563969890b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba0c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 0x563969890b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34570/0 (socket says 172.21.15.3:34570) 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7fab93e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969889a50 0x563969884d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7faba0c00640 1 -- 172.21.15.3:0/1901418112 learned_addr learned my addr 172.21.15.3:0/1901418112 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.611+0000 7fab9be00640 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x5639698a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7faba0c00640 1 -- 172.21.15.3:0/1901418112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 msgr2=0x5639698a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7faba0c00640 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x5639698a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7faba0c00640 1 -- 172.21.15.3:0/1901418112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969889a50 msgr2=0x563969884d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7faba0c00640 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969889a50 0x563969884d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7faba0c00640 1 -- 172.21.15.3:0/1901418112 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56396988f240 con 0x563969893280 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7faba0c00640 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 0x563969890b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab940113a0 tx=0x7fab94019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf608bfdd98ed6a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.612+0000 7fab8be00640 1 -- 172.21.15.3:0/1901418112 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9400fdd0 con 0x563969893280 2024-08-01T04:15:33.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7fab8be00640 1 -- 172.21.15.3:0/1901418112 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab94003970 con 0x563969893280 2024-08-01T04:15:33.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7fab8be00640 1 -- 172.21.15.3:0/1901418112 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940044f0 con 0x563969893280 2024-08-01T04:15:33.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 -- 172.21.15.3:0/1901418112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 msgr2=0x563969890b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 0x563969890b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab940113a0 tx=0x7fab94019000 comp rx=0 tx=0).stop 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 -- 172.21.15.3:0/1901418112 shutdown_connections 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969893280 0x563969890b60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563969889a50 0x563969884d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 --2- 172.21.15.3:0/1901418112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5639698884e0 0x5639698a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.613+0000 7faba57e2740 1 -- 172.21.15.3:0/1901418112 >> 172.21.15.3:0/1901418112 conn(0x563969850a00 msgr2=0x563969884610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.614+0000 7faba57e2740 1 -- 172.21.15.3:0/1901418112 shutdown_connections 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.614+0000 7faba57e2740 1 -- 172.21.15.3:0/1901418112 wait complete. 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.614+0000 7faba57e2740 1 -- start start 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.614+0000 7faba57e2740 1 Processor -- start 2024-08-01T04:15:33.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x5639698ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x56396988d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x56396988d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5639698a5420 con 0x5639698884e0 2024-08-01T04:15:33.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5639698a4d50 con 0x563969889a50 2024-08-01T04:15:33.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5639698a5af0 con 0x563969893280 2024-08-01T04:15:33.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba57e2740 10 client.0 _ll_register_callbacks cb 0x7fab84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7fab9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x5639698ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7faba0c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x56396988d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7fab9be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x5639698ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51308/0 (socket says 172.21.15.3:51308) 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7fab9be00640 1 -- 172.21.15.3:0/1313400667 learned_addr learned my addr 172.21.15.3:0/1313400667 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.615+0000 7fab93e00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x56396988d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba0c00640 1 -- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 msgr2=0x56396988d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba0c00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x56396988d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba0c00640 1 -- 172.21.15.3:0/1313400667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 msgr2=0x5639698ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:33.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba0c00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x5639698ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba0c00640 1 -- 172.21.15.3:0/1313400667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396988e120 con 0x563969893280 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7fab93e00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x56396988d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba0c00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x56396988d8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab94012010 tx=0x7fab94005a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842229efad5800cf server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7fab89600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7fab89600640 1 -- 172.21.15.3:0/1313400667 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94003d20 con 0x563969893280 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab94003970 con 0x563969893280 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.616+0000 7fab89600640 1 -- 172.21.15.3:0/1313400667 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fab940109b0 con 0x563969893280 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7fab89600640 1 -- 172.21.15.3:0/1313400667 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab94039070 con 0x563969893280 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 10 client.24880 fetch_fsmap learned FSMap version 25 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56396990e950 con 0x563969893280 2024-08-01T04:15:33.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7fab89600640 1 -- 172.21.15.3:0/1313400667 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab94003970 con 0x563969893280 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7fab89600640 1 -- 172.21.15.3:0/1313400667 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fab940354a0 con 0x563969893280 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 10 client.24880 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 10 client.24880 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 20 client.24880 populate_metadata read hostname 'smithi003' 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x563969913260 0x563969911f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 4 client.24880 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.617+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "status"}) -- 0x5639697f63c0 con 0x563969913260 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.618+0000 7fab9be00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x563969913260 0x563969911f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.618+0000 7fab9be00640 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x563969913260 0x563969911f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99258fb6fd689888 server_cookie=8ff50370fd9830df in_seq=0 out_seq=0 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.618+0000 7fab89600640 10 client.24880 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:15:33.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7fab89600640 1 -- 172.21.15.3:0/1313400667 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5639697f63c0 con 0x563969913260 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7fab89600640 10 client.24880 handle_command_reply: tid=0 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 client.24880 shutdown 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 msgr2=0x56396988d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x56396988d8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab94012010 tx=0x7fab94005a50 comp rx=0 tx=0).stop 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 shutdown_connections 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x563969913260 0x563969911f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563969893280 0x56396988d8f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563969889a50 0x56396988d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7fab89600640 0 client.24880 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 --2- 172.21.15.3:0/1313400667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5639698884e0 0x5639698ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 >> 172.21.15.3:0/1313400667 conn(0x563969850a00 msgr2=0x563969891900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:33.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 shutdown_connections 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 1 -- 172.21.15.3:0/1313400667 wait complete. 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.619+0000 7faba57e2740 20 client.24880 trim_cache size 0 max 16384 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24506, 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:15:33.624 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 86.024379354999994, 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 25, 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 88, 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 86, 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 125.49807750799999 2024-08-01T04:15:33.625 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fab800a2c90 msgr2=0x7fab800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fab800a2c90 0x7fab800a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab94005fd0 tx=0x7fab94009500 comp rx=0 tx=0).stop 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 msgr2=0x7fab9c19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c19e600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00ea60 tx=0x7fab8c00ef20 comp rx=0 tx=0).stop 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 shutdown_connections 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fab800a2c90 0x7fab800a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab9c10dd90 0x7fab9c1a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab9c1086b0 0x7fab9c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 --2- 172.21.15.3:0/3064752999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab9c107d50 0x7fab9c19e600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 >> 172.21.15.3:0/3064752999 conn(0x7fab9c0776b0 msgr2=0x7fab9c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 shutdown_connections 2024-08-01T04:15:33.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:33.620+0000 7faba1600640 1 -- 172.21.15.3:0/3064752999 wait complete. 2024-08-01T04:15:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:33 smithi160 ceph-mon[37224]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:33.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:33 smithi163 ceph-mon[37383]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:34.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.a status 2024-08-01T04:15:35.129 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:35.533 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:35 smithi003 ceph-mon[32262]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:35.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:35 smithi160 ceph-mon[37224]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:35.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:35 smithi163 ceph-mon[37383]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 -- 172.21.15.3:0/2958985103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384109b60 msgr2=0x7f1384109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 --2- 172.21.15.3:0/2958985103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384109b60 0x7f1384109f40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1378009930 tx=0x7f137802a420 comp rx=0 tx=0).stop 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 -- 172.21.15.3:0/2958985103 shutdown_connections 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 --2- 172.21.15.3:0/2958985103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384104470 0x7f13841109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 --2- 172.21.15.3:0/2958985103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f1384103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 --2- 172.21.15.3:0/2958985103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384109b60 0x7f1384109f40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 -- 172.21.15.3:0/2958985103 >> 172.21.15.3:0/2958985103 conn(0x7f13840ff750 msgr2=0x7f1384101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 -- 172.21.15.3:0/2958985103 shutdown_connections 2024-08-01T04:15:35.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.814+0000 7f138b400640 1 -- 172.21.15.3:0/2958985103 wait complete. 2024-08-01T04:15:35.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.815+0000 7f138b400640 1 Processor -- start 2024-08-01T04:15:35.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.815+0000 7f138b400640 1 -- start start 2024-08-01T04:15:35.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f138b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f138419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f138b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384104470 0x7f138419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f138b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 0x7f138419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f138b400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113ed0 con 0x7f1384104470 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f138b400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113d50 con 0x7f1384109b60 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f138b400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114050 con 0x7f1384103ad0 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f1383e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384104470 0x7f138419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f1388a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f138419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f1389400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 0x7f138419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f1388a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f138419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:45140/0 (socket says 172.21.15.3:45140) 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f1388a00640 1 -- 172.21.15.3:0/3889007070 learned_addr learned my addr 172.21.15.3:0/3889007070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.816+0000 7f1389400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 0x7f138419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34596/0 (socket says 172.21.15.3:34596) 2024-08-01T04:15:35.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1388a00640 1 -- 172.21.15.3:0/3889007070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 msgr2=0x7f138419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1388a00640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 0x7f138419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1388a00640 1 -- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384104470 msgr2=0x7f138419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1388a00640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384104470 0x7f138419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1388a00640 1 -- 172.21.15.3:0/3889007070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138419f3a0 con 0x7f1384103ad0 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1389400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 0x7f138419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1388a00640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f138419a3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1378009230 tx=0x7f1378004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1381600640 1 -- 172.21.15.3:0/3889007070 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137803f030 con 0x7f1384103ad0 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.817+0000 7f1381600640 1 -- 172.21.15.3:0/3889007070 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1378004440 con 0x7f1384103ad0 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.818+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138419f630 con 0x7f1384103ad0 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.818+0000 7f1381600640 1 -- 172.21.15.3:0/3889007070 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378035070 con 0x7f1384103ad0 2024-08-01T04:15:35.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.818+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a6f70 con 0x7f1384103ad0 2024-08-01T04:15:35.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.821+0000 7f138e008740 1 Processor -- start 2024-08-01T04:15:35.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.821+0000 7f138e008740 1 -- start start 2024-08-01T04:15:35.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.821+0000 7f1381600640 1 -- 172.21.15.3:0/3889007070 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1378042030 con 0x7f1384103ad0 2024-08-01T04:15:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.821+0000 7f138e008740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d3595add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.821+0000 7f138e008740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d3595b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f138e008740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d359660b0 0x563d359684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f138e008740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d358c1940 con 0x563d3595c880 2024-08-01T04:15:35.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f138e008740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563d358599a0 con 0x563d359660b0 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f138e008740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563d358bd900 con 0x563d3595b310 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1388a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d3595b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1389400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d3595add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1389400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d3595add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51328/0 (socket says 172.21.15.3:51328) 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1388a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d3595b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:45144/0 (socket says 172.21.15.3:45144) 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1389400640 1 -- 172.21.15.3:0/673981722 learned_addr learned my addr 172.21.15.3:0/673981722 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1383e00640 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d359660b0 0x563d359684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1383e00640 1 -- 172.21.15.3:0/673981722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 msgr2=0x563d3595b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1383e00640 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d3595b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1383e00640 1 -- 172.21.15.3:0/673981722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 msgr2=0x563d3595add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1383e00640 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d3595add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1383e00640 1 -- 172.21.15.3:0/673981722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d35847ef0 con 0x563d359660b0 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.822+0000 7f1389400640 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d3595add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1383e00640 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d359660b0 0x563d359684a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1374009bd0 tx=0x7f1374011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d566b2713d03d9b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1371c00640 1 -- 172.21.15.3:0/673981722 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374004230 con 0x563d359660b0 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1371c00640 1 -- 172.21.15.3:0/673981722 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374009e40 con 0x563d359660b0 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1371c00640 1 -- 172.21.15.3:0/673981722 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374004d50 con 0x563d359660b0 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1381600640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f136c0a2c90 0x7f136c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1381600640 1 -- 172.21.15.3:0/3889007070 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f13780ef4d0 con 0x7f1384103ad0 2024-08-01T04:15:35.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.823+0000 7f1389400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f136c0a2c90 0x7f136c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 -- 172.21.15.3:0/673981722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d359660b0 msgr2=0x563d359684a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d359660b0 0x563d359684a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1374009bd0 tx=0x7f1374011810 comp rx=0 tx=0).stop 2024-08-01T04:15:35.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 -- 172.21.15.3:0/673981722 shutdown_connections 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d359660b0 0x563d359684a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d3595b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 --2- 172.21.15.3:0/673981722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d3595add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 -- 172.21.15.3:0/673981722 >> 172.21.15.3:0/673981722 conn(0x563d35923a00 msgr2=0x563d359579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 -- 172.21.15.3:0/673981722 shutdown_connections 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f1389400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f136c0a2c90 0x7f136c0a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f137c009fd0 tx=0x7f137c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.824+0000 7f138e008740 1 -- 172.21.15.3:0/673981722 wait complete. 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.825+0000 7f138e008740 1 -- start start 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.825+0000 7f138e008740 1 Processor -- start 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d35962f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595c880 0x563d35963470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 0x563d359639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d35968db0 con 0x563d359660b0 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563d3587ca80 con 0x563d3595c880 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563d35968f30 con 0x563d3595b310 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f138e008740 10 client.0 _ll_register_callbacks cb 0x563d35845140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f1389400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 0x563d359639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f1389400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 0x563d359639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51334/0 (socket says 172.21.15.3:51334) 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f1389400640 1 -- 172.21.15.3:0/1490702993 learned_addr learned my addr 172.21.15.3:0/1490702993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f1388a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d35962f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.826+0000 7f1383e00640 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595c880 0x563d35963470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1389400640 1 -- 172.21.15.3:0/1490702993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 msgr2=0x563d35962f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1389400640 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d35962f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1389400640 1 -- 172.21.15.3:0/1490702993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595c880 msgr2=0x563d35963470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1389400640 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595c880 0x563d35963470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1389400640 1 -- 172.21.15.3:0/1490702993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d35860700 con 0x563d359660b0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1389400640 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 0x563d359639b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f137c012210 tx=0x7f137c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656223ad06a1873f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1366000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1366000640 1 -- 172.21.15.3:0/1490702993 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c010d40 con 0x563d359660b0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f1366000640 1 -- 172.21.15.3:0/1490702993 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f137c01e470 con 0x563d359660b0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.827+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d358ceeb0 con 0x563d359660b0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f1366000640 1 -- 172.21.15.3:0/1490702993 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f137c02f070 con 0x563d359660b0 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f138e008740 10 client.14781 fetch_fsmap learned FSMap version 25 2024-08-01T04:15:35.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d356332b0 con 0x563d359660b0 2024-08-01T04:15:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f1366000640 1 -- 172.21.15.3:0/1490702993 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563d358ceeb0 con 0x563d359660b0 2024-08-01T04:15:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f1366000640 1 -- 172.21.15.3:0/1490702993 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f137c027020 con 0x563d359660b0 2024-08-01T04:15:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f138e008740 10 client.14781 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:15:35.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f138e008740 10 client.14781 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f138e008740 20 client.14781 populate_metadata read hostname 'smithi003' 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.828+0000 7f138e008740 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d35976e80 0x563d359782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.829+0000 7f138e008740 4 client.14781 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.829+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d358c93c0 con 0x563d35976e80 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.829+0000 7f1388a00640 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d35976e80 0x563d359782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.829+0000 7f1388a00640 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d35976e80 0x563d359782f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d65c227b9c4a9691 server_cookie=6d2e2e363100b5d8 in_seq=0 out_seq=0 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.829+0000 7f1366000640 10 client.14781 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.830+0000 7f1366000640 1 -- 172.21.15.3:0/1490702993 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d358c93c0 con 0x563d35976e80 2024-08-01T04:15:35.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.830+0000 7f1366000640 10 client.14781 handle_command_reply: tid=0 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.830+0000 7f138e008740 1 client.14781 shutdown 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 msgr2=0x563d359639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 0x563d359639b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f137c012210 tx=0x7f137c004ab0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 shutdown_connections 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d35976e80 0x563d359782f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d359660b0 0x563d359639b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595c880 0x563d35963470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 --2- 172.21.15.3:0/1490702993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d3595b310 0x563d35962f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 >> 172.21.15.3:0/1490702993 conn(0x563d35923a00 msgr2=0x563d3595d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f1366000640 0 client.14781 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 shutdown_connections 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 1 -- 172.21.15.3:0/1490702993 wait complete. 2024-08-01T04:15:35.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.831+0000 7f138e008740 20 client.14781 trim_cache size 0 max 16384 2024-08-01T04:15:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.851+0000 7f138e008740 1 Processor -- start 2024-08-01T04:15:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.851+0000 7f138e008740 1 -- start start 2024-08-01T04:15:35.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.851+0000 7f138e008740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35962810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.851+0000 7f138e008740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35971900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f138e008740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d35960190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f138e008740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d35976630 con 0x563d3595c880 2024-08-01T04:15:35.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f138e008740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563d35976d00 con 0x563d3595b310 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f138e008740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563d359773d0 con 0x563d359660b0 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35971900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35971900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51336/0 (socket says 172.21.15.3:51336) 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 -- 172.21.15.3:0/2171550115 learned_addr learned my addr 172.21.15.3:0/2171550115 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1388a00640 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35962810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1389400640 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d35960190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 -- 172.21.15.3:0/2171550115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 msgr2=0x563d35960190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d35960190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 -- 172.21.15.3:0/2171550115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 msgr2=0x563d35962810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35962810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1383e00640 1 -- 172.21.15.3:0/2171550115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d359659e0 con 0x563d3595c880 2024-08-01T04:15:35.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.852+0000 7f1388a00640 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35962810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:15:35.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.853+0000 7f1383e00640 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35971900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1374009bd0 tx=0x7f1374009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd832ca567fc422 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.853+0000 7f1371200640 1 -- 172.21.15.3:0/2171550115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137401e040 con 0x563d3595c880 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.853+0000 7f1371200640 1 -- 172.21.15.3:0/2171550115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374004710 con 0x563d3595c880 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.853+0000 7f1371200640 1 -- 172.21.15.3:0/2171550115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13740246e0 con 0x563d3595c880 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 -- 172.21.15.3:0/2171550115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 msgr2=0x563d35971900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35971900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1374009bd0 tx=0x7f1374009b60 comp rx=0 tx=0).stop 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 -- 172.21.15.3:0/2171550115 shutdown_connections 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d35960190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35971900 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 --2- 172.21.15.3:0/2171550115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35962810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 -- 172.21.15.3:0/2171550115 >> 172.21.15.3:0/2171550115 conn(0x563d35923a00 msgr2=0x563d359711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:35.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 -- 172.21.15.3:0/2171550115 shutdown_connections 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.854+0000 7f138e008740 1 -- 172.21.15.3:0/2171550115 wait complete. 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 -- start start 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 Processor -- start 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35980c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35979ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d3597a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563d35978170 con 0x563d3595c880 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563d35977aa0 con 0x563d3595b310 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563d35978840 con 0x563d359660b0 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.855+0000 7f138e008740 10 client.0 _ll_register_callbacks cb 0x7f137c012900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35979ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35979ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51340/0 (socket says 172.21.15.3:51340) 2024-08-01T04:15:35.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 -- 172.21.15.3:0/2137987290 learned_addr learned my addr 172.21.15.3:0/2137987290 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:35.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1388a00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35980c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1389400640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d3597a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 -- 172.21.15.3:0/2137987290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 msgr2=0x563d3597a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d3597a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 -- 172.21.15.3:0/2137987290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 msgr2=0x563d35980c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35980c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 -- 172.21.15.3:0/2137987290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3597ac20 con 0x563d3595c880 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1389400640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d3597a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:15:35.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1388a00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35980c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1383e00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35979ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1374009710 tx=0x7f1374003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100cacf5a02bf194 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.856+0000 7f1366000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f1366000640 1 -- 172.21.15.3:0/2137987290 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374004150 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f1366000640 1 -- 172.21.15.3:0/2137987290 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f13740042f0 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1374004710 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f1366000640 1 -- 172.21.15.3:0/2137987290 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f137403c050 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f138e008740 10 client.14787 fetch_fsmap learned FSMap version 25 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d359e1590 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.857+0000 7f1366000640 1 -- 172.21.15.3:0/2137987290 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374004710 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f1366000640 1 -- 172.21.15.3:0/2137987290 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1374018070 con 0x563d3595c880 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f138e008740 10 client.14787 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:15:35.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f138e008740 10 client.14787 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f138e008740 20 client.14787 populate_metadata read hostname 'smithi003' 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f138e008740 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d359e5920 0x563d359e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f138e008740 4 client.14787 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "status"}) -- 0x563d35845f80 con 0x563d359e5920 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.858+0000 7f1388a00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d359e5920 0x563d359e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f1388a00640 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d359e5920 0x563d359e4630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b94f03a07fb4f92a server_cookie=9794cf3cf712441 in_seq=0 out_seq=0 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f1366000640 10 client.14787 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f1366000640 1 -- 172.21.15.3:0/2137987290 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d35845f80 con 0x563d359e5920 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f1366000640 10 client.14787 handle_command_reply: tid=0 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 client.14787 shutdown 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 msgr2=0x563d35979ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35979ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1374009710 tx=0x7f1374003ab0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 shutdown_connections 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x563d359e5920 0x563d359e4630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563d359660b0 0x563d3597a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563d3595c880 0x563d35979ff0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f1366000640 0 client.14787 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 --2- 172.21.15.3:0/2137987290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563d3595b310 0x563d35980c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.859+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 >> 172.21.15.3:0/2137987290 conn(0x563d35923a00 msgr2=0x563d3595d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.860+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 shutdown_connections 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.860+0000 7f138e008740 1 -- 172.21.15.3:0/2137987290 wait complete. 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.860+0000 7f138e008740 20 client.14787 trim_cache size 0 max 16384 2024-08-01T04:15:35.864 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24520, 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 89.274683464999995, 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 25, 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 88, 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 86, 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 117.179779977 2024-08-01T04:15:35.865 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:15:35.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f136c0a2c90 msgr2=0x7f136c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f136c0a2c90 0x7f136c0a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f137c009fd0 tx=0x7f137c009500 comp rx=0 tx=0).stop 2024-08-01T04:15:35.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 msgr2=0x7f138419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:35.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f138419a3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1378009230 tx=0x7f1378004040 comp rx=0 tx=0).stop 2024-08-01T04:15:35.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 shutdown_connections 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f136c0a2c90 0x7f136c0a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1384109b60 0x7f138419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1384104470 0x7f138419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 --2- 172.21.15.3:0/3889007070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1384103ad0 0x7f138419a3f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 >> 172.21.15.3:0/3889007070 conn(0x7f13840ff750 msgr2=0x7f1384100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 shutdown_connections 2024-08-01T04:15:35.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:35.861+0000 7f138b400640 1 -- 172.21.15.3:0/3889007070 wait complete. 2024-08-01T04:15:36.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-08-01T04:15:37.392 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:37.570 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:37 smithi003 ceph-mon[32262]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:37.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:37 smithi160 ceph-mon[37224]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:37.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:37 smithi163 ceph-mon[37383]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.068+0000 7f02e2400640 1 -- 172.21.15.3:0/2744437303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc109900 msgr2=0x7f02dc109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.068+0000 7f02e2400640 1 --2- 172.21.15.3:0/2744437303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc109900 0x7f02dc109ce0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009930 tx=0x7f02cc02a420 comp rx=0 tx=0).stop 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 -- 172.21.15.3:0/2744437303 shutdown_connections 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 --2- 172.21.15.3:0/2744437303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 0x7f02dc1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 --2- 172.21.15.3:0/2744437303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02dc103900 0x7f02dc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 --2- 172.21.15.3:0/2744437303 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc109900 0x7f02dc109ce0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 -- 172.21.15.3:0/2744437303 >> 172.21.15.3:0/2744437303 conn(0x7f02dc0ff600 msgr2=0x7f02dc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:38.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 -- 172.21.15.3:0/2744437303 shutdown_connections 2024-08-01T04:15:38.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.069+0000 7f02e2400640 1 -- 172.21.15.3:0/2744437303 wait complete. 2024-08-01T04:15:38.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.070+0000 7f02e2400640 1 Processor -- start 2024-08-01T04:15:38.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.070+0000 7f02e2400640 1 -- start start 2024-08-01T04:15:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e2400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 0x7f02dc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e2400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 0x7f02dc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e2400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02dc109900 0x7f02dc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e2400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc113d50 con 0x7f02dc103900 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e2400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc113bd0 con 0x7f02dc1042a0 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e2400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc113ed0 con 0x7f02dc109900 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e1000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 0x7f02dc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e1000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 0x7f02dc19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51364/0 (socket says 172.21.15.3:51364) 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e1000640 1 -- 172.21.15.3:0/2145139713 learned_addr learned my addr 172.21.15.3:0/2145139713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02d3e00640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 0x7f02dc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.071+0000 7f02e1a00640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02dc109900 0x7f02dc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e1000640 1 -- 172.21.15.3:0/2145139713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02dc109900 msgr2=0x7f02dc19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e1000640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02dc109900 0x7f02dc19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e1000640 1 -- 172.21.15.3:0/2145139713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 msgr2=0x7f02dc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e1000640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 0x7f02dc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e1000640 1 -- 172.21.15.3:0/2145139713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc19f1c0 con 0x7f02dc103900 2024-08-01T04:15:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02d3e00640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 0x7f02dc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e1000640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 0x7f02dc19a210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02cc02a940 tx=0x7f02cc003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02da000640 1 -- 172.21.15.3:0/2145139713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc03f070 con 0x7f02dc103900 2024-08-01T04:15:38.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02da000640 1 -- 172.21.15.3:0/2145139713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc031c90 con 0x7f02dc103900 2024-08-01T04:15:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02da000640 1 -- 172.21.15.3:0/2145139713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc035030 con 0x7f02dc103900 2024-08-01T04:15:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.072+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc19f450 con 0x7f02dc103900 2024-08-01T04:15:38.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.073+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc1a6d90 con 0x7f02dc103900 2024-08-01T04:15:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.076+0000 7f02da000640 1 -- 172.21.15.3:0/2145139713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f02cc042070 con 0x7f02dc103900 2024-08-01T04:15:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.076+0000 7f02e662d740 1 Processor -- start 2024-08-01T04:15:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.076+0000 7f02e662d740 1 -- start start 2024-08-01T04:15:38.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e662d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e662d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e662d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e662d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fd57940 con 0x55ad1fdfc280 2024-08-01T04:15:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e662d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fcef9a0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e662d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fd53900 con 0x55ad1fdf14e0 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e1a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e1a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34638/0 (socket says 172.21.15.3:34638) 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e1000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.077+0000 7f02e1a00640 1 -- 172.21.15.3:0/86084762 learned_addr learned my addr 172.21.15.3:0/86084762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1a00640 1 -- 172.21.15.3:0/86084762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 msgr2=0x55ad1fdf1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1a00640 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1a00640 1 -- 172.21.15.3:0/86084762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 msgr2=0x55ad1fdfe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1a00640 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdfe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1a00640 1 -- 172.21.15.3:0/86084762 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1fcddef0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02da000640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02c00a2c90 0x7f02c00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02da000640 1 -- 172.21.15.3:0/2145139713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f02cc0efb50 con 0x7f02dc103900 2024-08-01T04:15:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1a00640 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf0fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009f60 tx=0x7f02d4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcf55e4c2e47861 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:38.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02e1000640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02c00a2c90 0x7f02c00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02d2000640 1 -- 172.21.15.3:0/86084762 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d401e070 con 0x55ad1fdf2a50 2024-08-01T04:15:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02d2000640 1 -- 172.21.15.3:0/86084762 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d4003c30 con 0x55ad1fdf2a50 2024-08-01T04:15:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.078+0000 7f02d2000640 1 -- 172.21.15.3:0/86084762 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d400f9c0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 -- 172.21.15.3:0/86084762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 msgr2=0x55ad1fdf0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e1000640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02c00a2c90 0x7f02c00a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0c4910 tx=0x7f02cc0f3600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:38.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf0fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009f60 tx=0x7f02d4009f90 comp rx=0 tx=0).stop 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 -- 172.21.15.3:0/86084762 shutdown_connections 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdfe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 --2- 172.21.15.3:0/86084762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf0fa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.079+0000 7f02e662d740 1 -- 172.21.15.3:0/86084762 >> 172.21.15.3:0/86084762 conn(0x55ad1fdb9a00 msgr2=0x55ad1fdedb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.080+0000 7f02e662d740 1 -- 172.21.15.3:0/86084762 shutdown_connections 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.080+0000 7f02e662d740 1 -- 172.21.15.3:0/86084762 wait complete. 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.080+0000 7f02e662d740 1 -- start start 2024-08-01T04:15:38.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.080+0000 7f02e662d740 1 Processor -- start 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fdfef80 con 0x55ad1fdf2a50 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fd12a80 con 0x55ad1fdf14e0 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fdff100 con 0x55ad1fdfc280 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e662d740 10 client.0 _ll_register_callbacks cb 0x55ad1fcdb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e1a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51384/0 (socket says 172.21.15.3:51384) 2024-08-01T04:15:38.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e1a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34646/0 (socket says 172.21.15.3:34646) 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02d3e00640 1 -- 172.21.15.3:0/3803278446 learned_addr learned my addr 172.21.15.3:0/3803278446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.081+0000 7f02e1a00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02d3e00640 1 -- 172.21.15.3:0/3803278446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdfc280 msgr2=0x55ad1fdf9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02d3e00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02d3e00640 1 -- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf14e0 msgr2=0x55ad1fdf9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02d3e00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02d3e00640 1 -- 172.21.15.3:0/3803278446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1fcf6700 con 0x55ad1fdf2a50 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02e1a00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02d3e00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf9770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02c4012d20 tx=0x7f02c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cf43dec7a524ec server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02caa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.082+0000 7f02caa00640 1 -- 172.21.15.3:0/3803278446 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4012110 con 0x55ad1fdf2a50 2024-08-01T04:15:38.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02caa00640 1 -- 172.21.15.3:0/3803278446 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f02c4004050 con 0x55ad1fdf2a50 2024-08-01T04:15:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d4003c30 con 0x55ad1fdf2a50 2024-08-01T04:15:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02caa00640 1 -- 172.21.15.3:0/3803278446 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02c4022070 con 0x55ad1fdf2a50 2024-08-01T04:15:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02e662d740 10 client.14799 fetch_fsmap learned FSMap version 25 2024-08-01T04:15:38.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad1fac92b0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02caa00640 1 -- 172.21.15.3:0/3803278446 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02d4003c30 con 0x55ad1fdf2a50 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.083+0000 7f02caa00640 1 -- 172.21.15.3:0/3803278446 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02c4010430 con 0x55ad1fdf2a50 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e662d740 10 client.14799 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e662d740 10 client.14799 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e662d740 20 client.14799 populate_metadata read hostname 'smithi003' 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e662d740 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe0d160 0x55ad1fe0e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e662d740 4 client.14799 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad1fd64eb0 con 0x55ad1fe0d160 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.084+0000 7f02e1a00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe0d160 0x55ad1fe0e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.085+0000 7f02e1a00640 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe0d160 0x55ad1fe0e5d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8f914078d5c628 server_cookie=14a4ea945bb62341 in_seq=0 out_seq=0 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.085+0000 7f02caa00640 10 client.14799 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:15:38.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02caa00640 1 -- 172.21.15.3:0/3803278446 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad1fd64eb0 con 0x55ad1fe0d160 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02caa00640 10 client.14799 handle_command_reply: tid=0 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 client.14799 shutdown 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 msgr2=0x55ad1fdf9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf9770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02c4012d20 tx=0x7f02c400f790 comp rx=0 tx=0).stop 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 shutdown_connections 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe0d160 0x55ad1fe0e5d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf9770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 --2- 172.21.15.3:0/3803278446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf14e0 0x55ad1fdf9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 >> 172.21.15.3:0/3803278446 conn(0x55ad1fdb9a00 msgr2=0x55ad1fdf3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02caa00640 0 client.14799 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:15:38.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 shutdown_connections 2024-08-01T04:15:38.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.086+0000 7f02e662d740 1 -- 172.21.15.3:0/3803278446 wait complete. 2024-08-01T04:15:38.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.087+0000 7f02e662d740 20 client.14799 trim_cache size 0 max 16384 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 Processor -- start 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 -- start start 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe0d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fe0c910 con 0x55ad1fdfc280 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fe0cfe0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.107+0000 7f02e662d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fe0d6b0 con 0x55ad1fdf14e0 2024-08-01T04:15:38.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34658/0 (socket says 172.21.15.3:34658) 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 -- 172.21.15.3:0/2691173788 learned_addr learned my addr 172.21.15.3:0/2691173788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02e1a00640 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe0d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02e1a00640 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 -- 172.21.15.3:0/2691173788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 msgr2=0x55ad1fe0d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe0d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 -- 172.21.15.3:0/2691173788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 msgr2=0x55ad1fdf8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 -- 172.21.15.3:0/2691173788 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1fdf4030 con 0x55ad1fdf2a50 2024-08-01T04:15:38.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02e1a00640 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:15:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.108+0000 7f02d3e00640 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf7b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02c4012cf0 tx=0x7f02c4005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314e66b0df36f80a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02d1600640 1 -- 172.21.15.3:0/2691173788 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c402b020 con 0x55ad1fdf2a50 2024-08-01T04:15:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02d1600640 1 -- 172.21.15.3:0/2691173788 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02c4005590 con 0x55ad1fdf2a50 2024-08-01T04:15:38.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02d1600640 1 -- 172.21.15.3:0/2691173788 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40035f0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 -- 172.21.15.3:0/2691173788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 msgr2=0x55ad1fdf7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf7b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02c4012cf0 tx=0x7f02c4005db0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 -- 172.21.15.3:0/2691173788 shutdown_connections 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdfc280 0x55ad1fdf8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdf2a50 0x55ad1fdf7b00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 --2- 172.21.15.3:0/2691173788 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe0d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.109+0000 7f02e662d740 1 -- 172.21.15.3:0/2691173788 >> 172.21.15.3:0/2691173788 conn(0x55ad1fdb9a00 msgr2=0x55ad1fdf4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.110+0000 7f02e662d740 1 -- 172.21.15.3:0/2691173788 shutdown_connections 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.110+0000 7f02e662d740 1 -- 172.21.15.3:0/2691173788 wait complete. 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.110+0000 7f02e662d740 1 -- start start 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.110+0000 7f02e662d740 1 Processor -- start 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe16a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf2a50 0x55ad1fe16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdfc280 0x55ad1fe0fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fe0e450 con 0x55ad1fdf14e0 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fe0dd80 con 0x55ad1fdfc280 2024-08-01T04:15:38.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ad1fe0eb20 con 0x55ad1fdf2a50 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e662d740 10 client.0 _ll_register_callbacks cb 0x7f02c4009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe16a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02d3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf2a50 0x55ad1fe16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe16a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51392/0 (socket says 172.21.15.3:51392) 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e1a00640 1 -- 172.21.15.3:0/1354924993 learned_addr learned my addr 172.21.15.3:0/1354924993 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02e1a00640 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdfc280 0x55ad1fe0fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02d3e00640 1 -- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdfc280 msgr2=0x55ad1fe0fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.111+0000 7f02d3e00640 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdfc280 0x55ad1fe0fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02d3e00640 1 -- 172.21.15.3:0/1354924993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf14e0 msgr2=0x55ad1fe16a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02d3e00640 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe16a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02d3e00640 1 -- 172.21.15.3:0/1354924993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1fe105e0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02d3e00640 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf2a50 0x55ad1fe16fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009c90 tx=0x7f02c4012340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ab9974bf24004f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02caa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02caa00640 1 -- 172.21.15.3:0/1354924993 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4034070 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c4005590 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02caa00640 1 -- 172.21.15.3:0/1354924993 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f02c4005040 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02caa00640 1 -- 172.21.15.3:0/1354924993 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02c4040050 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02e662d740 10 client.24916 fetch_fsmap learned FSMap version 25 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.112+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad1fe14fb0 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02caa00640 1 -- 172.21.15.3:0/1354924993 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02c4005590 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02caa00640 1 -- 172.21.15.3:0/1354924993 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02c4027480 con 0x55ad1fdf2a50 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02e662d740 10 client.24916 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02e662d740 10 client.24916 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-01T04:15:38.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02e662d740 20 client.24916 populate_metadata read hostname 'smithi003' 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02e662d740 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe7bae0 0x55ad1fe7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02e662d740 4 client.24916 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.113+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "status"}) -- 0x55ad1fd5f3c0 con 0x55ad1fe7bae0 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.114+0000 7f02e1a00640 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe7bae0 0x55ad1fe7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.114+0000 7f02e1a00640 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe7bae0 0x55ad1fe7a7f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0bd39fb4b98cd0 server_cookie=24f44402ad69996d in_seq=0 out_seq=0 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.114+0000 7f02caa00640 10 client.24916 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.114+0000 7f02caa00640 1 -- 172.21.15.3:0/1354924993 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ad1fd5f3c0 con 0x55ad1fe7bae0 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.114+0000 7f02caa00640 10 client.24916 handle_command_reply: tid=0 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.114+0000 7f02e662d740 1 client.24916 shutdown 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf2a50 msgr2=0x55ad1fe16fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf2a50 0x55ad1fe16fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009c90 tx=0x7f02c4012340 comp rx=0 tx=0).stop 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 shutdown_connections 2024-08-01T04:15:38.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55ad1fe7bae0 0x55ad1fe7a7f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ad1fdfc280 0x55ad1fe0fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ad1fdf2a50 0x55ad1fe16fc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 --2- 172.21.15.3:0/1354924993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ad1fdf14e0 0x55ad1fe16a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02caa00640 0 client.24916 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 >> 172.21.15.3:0/1354924993 conn(0x55ad1fdb9a00 msgr2=0x55ad1fdf7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 shutdown_connections 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 1 -- 172.21.15.3:0/1354924993 wait complete. 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.115+0000 7f02e662d740 20 client.24916 trim_cache size 0 max 16384 2024-08-01T04:15:38.119 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24548, 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 101.59489444800001, 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 25, 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 88, 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 86, 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 116.637500011 2024-08-01T04:15:38.120 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02c00a2c90 msgr2=0x7f02c00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02c00a2c90 0x7f02c00a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0c4910 tx=0x7f02cc0f3600 comp rx=0 tx=0).stop 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 msgr2=0x7f02dc19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 0x7f02dc19a210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02cc02a940 tx=0x7f02cc003b20 comp rx=0 tx=0).stop 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 shutdown_connections 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f02c00a2c90 0x7f02c00a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02dc109900 0x7f02dc19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02dc1042a0 0x7f02dc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 --2- 172.21.15.3:0/2145139713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02dc103900 0x7f02dc19a210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 >> 172.21.15.3:0/2145139713 conn(0x7f02dc0ff600 msgr2=0x7f02dc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 shutdown_connections 2024-08-01T04:15:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:38.116+0000 7f02e2400640 1 -- 172.21.15.3:0/2145139713 wait complete. 2024-08-01T04:15:39.179 INFO:tasks.cephadm:Setup complete, yielding 2024-08-01T04:15:39.180 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-01T04:15:39.190 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:15:39.190 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch status' 2024-08-01T04:15:39.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:39 smithi003 ceph-mon[32262]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:39.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:39 smithi160 ceph-mon[37224]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:39.664 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:39.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:39 smithi163 ceph-mon[37383]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.308+0000 7fc871e00640 1 -- 172.21.15.3:0/3604604652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c102970 msgr2=0x7fc86c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.308+0000 7fc871e00640 1 --2- 172.21.15.3:0/3604604652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c102970 0x7fc86c102df0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc858009860 tx=0x7fc85802a340 comp rx=0 tx=0).stop 2024-08-01T04:15:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 -- 172.21.15.3:0/3604604652 shutdown_connections 2024-08-01T04:15:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 --2- 172.21.15.3:0/3604604652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c103330 0x7fc86c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 --2- 172.21.15.3:0/3604604652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c102970 0x7fc86c102df0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 --2- 172.21.15.3:0/3604604652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 -- 172.21.15.3:0/3604604652 >> 172.21.15.3:0/3604604652 conn(0x7fc86c0fcf20 msgr2=0x7fc86c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 -- 172.21.15.3:0/3604604652 shutdown_connections 2024-08-01T04:15:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.309+0000 7fc871e00640 1 -- 172.21.15.3:0/3604604652 wait complete. 2024-08-01T04:15:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.310+0000 7fc871e00640 1 Processor -- start 2024-08-01T04:15:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.310+0000 7fc871e00640 1 -- start start 2024-08-01T04:15:40.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.310+0000 7fc871e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.310+0000 7fc871e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c102970 0x7fc86c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.310+0000 7fc871e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c103330 0x7fc86c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc871e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c10d3d0 con 0x7fc86c101770 2024-08-01T04:15:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc871e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c10d250 con 0x7fc86c103330 2024-08-01T04:15:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc871e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c10d550 con 0x7fc86c102970 2024-08-01T04:15:40.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51398/0 (socket says 172.21.15.3:51398) 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 -- 172.21.15.3:0/2943105873 learned_addr learned my addr 172.21.15.3:0/2943105873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc86be00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c102970 0x7fc86c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc871400640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c103330 0x7fc86c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 -- 172.21.15.3:0/2943105873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c102970 msgr2=0x7fc86c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c102970 0x7fc86c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 -- 172.21.15.3:0/2943105873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c103330 msgr2=0x7fc86c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c103330 0x7fc86c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc870a00640 1 -- 172.21.15.3:0/2943105873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c19f680 con 0x7fc86c101770 2024-08-01T04:15:40.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.311+0000 7fc86be00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c102970 0x7fc86c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.312+0000 7fc870a00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c197a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e9f0 tx=0x7fc86000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.312+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000ccc0 con 0x7fc86c101770 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.312+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8600035f0 con 0x7fc86c101770 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.312+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c19f970 con 0x7fc86c101770 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.312+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000ce60 con 0x7fc86c101770 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.312+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c19fe80 con 0x7fc86c101770 2024-08-01T04:15:40.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.313+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c101ba0 con 0x7fc86c101770 2024-08-01T04:15:40.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.315+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc8600037a0 con 0x7fc86c101770 2024-08-01T04:15:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.317+0000 7fc869600640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8540a2c40 0x7fc8540a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:40.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.317+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7fc8600cf490 con 0x7fc86c101770 2024-08-01T04:15:40.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.317+0000 7fc86be00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8540a2c40 0x7fc8540a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:40.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.318+0000 7fc86be00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8540a2c40 0x7fc8540a5100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc85802a930 tx=0x7fc858030570 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.321+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc86c101ba0 con 0x7fc86c101770 2024-08-01T04:15:40.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.533+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc86c062360 con 0x7fc8540a2c40 2024-08-01T04:15:40.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.535+0000 7fc869600640 1 -- 172.21.15.3:0/2943105873 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc86c062360 con 0x7fc8540a2c40 2024-08-01T04:15:40.536 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2024-08-01T04:15:40.537 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2024-08-01T04:15:40.537 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2024-08-01T04:15:40.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8540a2c40 msgr2=0x7fc8540a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:40.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8540a2c40 0x7fc8540a5100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc85802a930 tx=0x7fc858030570 comp rx=0 tx=0).stop 2024-08-01T04:15:40.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 msgr2=0x7fc86c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:40.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c197a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e9f0 tx=0x7fc86000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 shutdown_connections 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fc8540a2c40 0x7fc8540a5100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc86c103330 0x7fc86c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c102970 0x7fc86c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 --2- 172.21.15.3:0/2943105873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc86c101770 0x7fc86c197a50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 >> 172.21.15.3:0/2943105873 conn(0x7fc86c0fcf20 msgr2=0x7fc86c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.539+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 shutdown_connections 2024-08-01T04:15:40.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:40.540+0000 7fc871e00640 1 -- 172.21.15.3:0/2943105873 wait complete. 2024-08-01T04:15:41.509 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-01T04:15:41.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:41 smithi003 ceph-mon[32262]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:41.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:41 smithi003 ceph-mon[32262]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:41.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:41 smithi160 ceph-mon[37224]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:41.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:41 smithi160 ceph-mon[37224]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:41.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:41 smithi163 ceph-mon[37383]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:41.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:41 smithi163 ceph-mon[37383]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:41.985 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.654+0000 7f44d7e00640 1 -- 172.21.15.3:0/247547922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d0075ed0 msgr2=0x7f44d0074330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.654+0000 7f44d7e00640 1 --2- 172.21.15.3:0/247547922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d0075ed0 0x7f44d0074330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009ca0 tx=0x7f44c002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 -- 172.21.15.3:0/247547922 shutdown_connections 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 --2- 172.21.15.3:0/247547922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44d01097b0 0x7f44d010bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 --2- 172.21.15.3:0/247547922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 --2- 172.21.15.3:0/247547922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d0075ed0 0x7f44d0074330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 -- 172.21.15.3:0/247547922 >> 172.21.15.3:0/247547922 conn(0x7f44d00fd0f0 msgr2=0x7f44d00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 -- 172.21.15.3:0/247547922 shutdown_connections 2024-08-01T04:15:42.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.655+0000 7f44d7e00640 1 -- 172.21.15.3:0/247547922 wait complete. 2024-08-01T04:15:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 Processor -- start 2024-08-01T04:15:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 -- start start 2024-08-01T04:15:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44d0075ed0 0x7f44d019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:42.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d01097b0 0x7f44d01a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010f800 con 0x7f44d01097b0 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010f680 con 0x7f44d0074870 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010f980 con 0x7f44d0075ed0 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:44310/0 (socket says 172.21.15.3:44310) 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.656+0000 7f44d5400640 1 -- 172.21.15.3:0/1204895215 learned_addr learned my addr 172.21.15.3:0/1204895215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d01097b0 0x7f44d01a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d4a00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44d0075ed0 0x7f44d019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 -- 172.21.15.3:0/1204895215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44d0075ed0 msgr2=0x7f44d019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44d0075ed0 0x7f44d019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 -- 172.21.15.3:0/1204895215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 msgr2=0x7f44d0199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 -- 172.21.15.3:0/1204895215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d01a1a20 con 0x7f44d01097b0 2024-08-01T04:15:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5400640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.657+0000 7f44d5e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d01097b0 0x7f44d01a13b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44c800e970 tx=0x7f44c800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.658+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c800ccc0 con 0x7f44d01097b0 2024-08-01T04:15:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.658+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44c8005020 con 0x7f44d01097b0 2024-08-01T04:15:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.658+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8004a00 con 0x7f44d01097b0 2024-08-01T04:15:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.658+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d01a1d10 con 0x7f44d01097b0 2024-08-01T04:15:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.658+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d01a21a0 con 0x7f44d01097b0 2024-08-01T04:15:42.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.659+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d01085c0 con 0x7f44d01097b0 2024-08-01T04:15:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.662+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f44c80051d0 con 0x7f44d01097b0 2024-08-01T04:15:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.663+0000 7f44ce000640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f44b40a2cc0 0x7f44b40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.663+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f44c80cf0c0 con 0x7f44d01097b0 2024-08-01T04:15:42.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.663+0000 7f44d5400640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f44b40a2cc0 0x7f44b40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:42.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.664+0000 7f44d5400640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f44b40a2cc0 0x7f44b40a5180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f44c0004ab0 tx=0x7f44c0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:42.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.667+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f44c809be40 con 0x7f44d01097b0 2024-08-01T04:15:42.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.873+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f44d0062360 con 0x7f44b40a2cc0 2024-08-01T04:15:42.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.895+0000 7f44ce000640 1 -- 172.21.15.3:0/1204895215 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f44d0062360 con 0x7f44b40a2cc0 2024-08-01T04:15:42.896 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.a smithi003 running (2m) 26s ago 2m 16.3M - 19.1.0-1223-gf9911e17 93d93c74a2cc e37918a6ea2b 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.b smithi160 running (2m) 50s ago 2m 28.7M - 19.1.0-1223-gf9911e17 93d93c74a2cc 03633524b636 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.c smithi163 running (2m) 50s ago 2m 16.8M - 19.1.0-1223-gf9911e17 93d93c74a2cc 9eae5ad1084f 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.d smithi003 running (2m) 26s ago 2m 12.7M - 19.1.0-1223-gf9911e17 93d93c74a2cc aeb472cdb365 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.e smithi160 running (2m) 50s ago 2m 18.9M - 19.1.0-1223-gf9911e17 93d93c74a2cc 3ef621ec6fb5 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.f smithi163 running (2m) 50s ago 2m 14.1M - 19.1.0-1223-gf9911e17 93d93c74a2cc 6b97cc6642de 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.g smithi003 running (2m) 26s ago 2m 12.6M - 19.1.0-1223-gf9911e17 93d93c74a2cc d57023685374 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.h smithi160 running (2m) 50s ago 2m 19.9M - 19.1.0-1223-gf9911e17 93d93c74a2cc 05287f9082e1 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.i smithi163 running (2m) 50s ago 2m 17.9M - 19.1.0-1223-gf9911e17 93d93c74a2cc 7854fce8c994 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.j smithi003 running (2m) 26s ago 2m 14.9M - 19.1.0-1223-gf9911e17 93d93c74a2cc cf5e3f2346f8 2024-08-01T04:15:42.897 INFO:teuthology.orchestra.run.smithi003.stdout:mds.k smithi160 running (2m) 50s ago 2m 14.5M - 19.1.0-1223-gf9911e17 93d93c74a2cc 83e17d5f6434 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mds.l smithi163 running (2m) 50s ago 2m 14.5M - 19.1.0-1223-gf9911e17 93d93c74a2cc 5a666cacb592 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi003 *:9283,8765 running (9m) 26s ago 9m 509M - 19.1.0-1223-gf9911e17 93d93c74a2cc ddbc642f976a 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi160 *:8443,8765 running (7m) 50s ago 7m 442M - 19.1.0-1223-gf9911e17 93d93c74a2cc 958a0df1a180 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi163 *:8443,8765 running (7m) 50s ago 7m 441M - 19.1.0-1223-gf9911e17 93d93c74a2cc 79730b59046b 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (9m) 26s ago 9m 47.3M 2048M 19.1.0-1223-gf9911e17 93d93c74a2cc d8af0539c3ef 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi160 running (8m) 50s ago 8m 41.7M 2048M 19.1.0-1223-gf9911e17 93d93c74a2cc 2ec6feaf7ed1 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi163 running (8m) 50s ago 8m 41.0M 2048M 19.1.0-1223-gf9911e17 93d93c74a2cc 56cb51c00c8f 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (7m) 26s ago 7m 56.2M 4273M 19.1.0-1223-gf9911e17 93d93c74a2cc 8d0671a64106 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi160 running (7m) 50s ago 7m 47.6M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc 42e3ab4677aa 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi163 running (6m) 50s ago 6m 48.8M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc 611b60314c91 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 smithi003 running (6m) 26s ago 6m 41.4M 4273M 19.1.0-1223-gf9911e17 93d93c74a2cc 4d0a0f1a7e69 2024-08-01T04:15:42.898 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 smithi160 running (5m) 50s ago 5m 52.9M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc d8b75ceafc42 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 smithi163 running (5m) 50s ago 5m 56.3M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc dfd831331f2b 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 smithi003 running (5m) 26s ago 4m 40.6M 4273M 19.1.0-1223-gf9911e17 93d93c74a2cc 5db8b66546d9 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 smithi160 running (4m) 50s ago 4m 43.5M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc b0b1eb800012 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 smithi163 running (4m) 50s ago 4m 53.9M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc 17f2a7b308af 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 smithi003 running (3m) 26s ago 3m 49.2M 4273M 19.1.0-1223-gf9911e17 93d93c74a2cc 69097757d780 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 smithi160 running (3m) 50s ago 3m 52.7M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc 545b8eaab26f 2024-08-01T04:15:42.899 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 smithi163 running (2m) 50s ago 2m 41.1M 4272M 19.1.0-1223-gf9911e17 93d93c74a2cc a5e21f1d2f59 2024-08-01T04:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.898+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f44b40a2cc0 msgr2=0x7f44b40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.898+0000 7f44d7e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f44b40a2cc0 0x7f44b40a5180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f44c0004ab0 tx=0x7f44c0004a40 comp rx=0 tx=0).stop 2024-08-01T04:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.898+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d01097b0 msgr2=0x7f44d01a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.898+0000 7f44d7e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d01097b0 0x7f44d01a13b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44c800e970 tx=0x7f44c800ee30 comp rx=0 tx=0).stop 2024-08-01T04:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 shutdown_connections 2024-08-01T04:15:42.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f44b40a2cc0 0x7f44b40a5180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44d01097b0 0x7f44d01a13b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44d0075ed0 0x7f44d019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 --2- 172.21.15.3:0/1204895215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44d0074870 0x7f44d0199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 >> 172.21.15.3:0/1204895215 conn(0x7f44d00fd0f0 msgr2=0x7f44d0105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 shutdown_connections 2024-08-01T04:15:42.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:42.899+0000 7f44d7e00640 1 -- 172.21.15.3:0/1204895215 wait complete. 2024-08-01T04:15:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:43 smithi160 ceph-mon[37224]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:43 smithi160 ceph-mon[37224]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:43.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:43 smithi003 ceph-mon[32262]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:43.781 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:43 smithi003 ceph-mon[32262]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:43.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:43 smithi163 ceph-mon[37383]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-01T04:15:43.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:43 smithi163 ceph-mon[37383]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:43.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-01T04:15:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 -- 172.21.15.3:0/3916226746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108f70 msgr2=0x7f27541093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 --2- 172.21.15.3:0/3916226746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108f70 0x7f27541093f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2740009900 tx=0x7f274002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 -- 172.21.15.3:0/3916226746 shutdown_connections 2024-08-01T04:15:45.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 --2- 172.21.15.3:0/3916226746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2754109930 0x7f27541101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 --2- 172.21.15.3:0/3916226746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108f70 0x7f27541093f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 --2- 172.21.15.3:0/3916226746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754108130 0x7f2754108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 -- 172.21.15.3:0/3916226746 >> 172.21.15.3:0/3916226746 conn(0x7f2754076f30 msgr2=0x7f2754079370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 -- 172.21.15.3:0/3916226746 shutdown_connections 2024-08-01T04:15:45.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.003+0000 7f2759200640 1 -- 172.21.15.3:0/3916226746 wait complete. 2024-08-01T04:15:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 Processor -- start 2024-08-01T04:15:45.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 -- start start 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 0x7f275419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754108f70 0x7f275419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2754109930 0x7f27541a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2754113a20 con 0x7f2754108130 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f27541138a0 con 0x7f2754108f70 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.004+0000 7f2759200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2754113ba0 con 0x7f2754109930 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 0x7f275419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 0x7f275419e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48528/0 (socket says 172.21.15.3:48528) 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 -- 172.21.15.3:0/237192576 learned_addr learned my addr 172.21.15.3:0/237192576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2752a00640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754108f70 0x7f275419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753e00640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2754109930 0x7f27541a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:45.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 -- 172.21.15.3:0/237192576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2754109930 msgr2=0x7f27541a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:45.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2754109930 0x7f27541a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 -- 172.21.15.3:0/237192576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754108f70 msgr2=0x7f275419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:45.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754108f70 0x7f275419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.005+0000 7f2753400640 1 -- 172.21.15.3:0/237192576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27541a5ca0 con 0x7f2754108130 2024-08-01T04:15:45.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.006+0000 7f2753400640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 0x7f275419e070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f274400d950 tx=0x7f274400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.006+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744017020 con 0x7f2754108130 2024-08-01T04:15:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.006+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2744009c30 con 0x7f2754108130 2024-08-01T04:15:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.006+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27541a5f90 con 0x7f2754108130 2024-08-01T04:15:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.006+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274400f720 con 0x7f2754108130 2024-08-01T04:15:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.006+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27541a63e0 con 0x7f2754108130 2024-08-01T04:15:45.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.007+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275410f120 con 0x7f2754108130 2024-08-01T04:15:45.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.010+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f274401a020 con 0x7f2754108130 2024-08-01T04:15:45.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.011+0000 7f274be00640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27380a2c40 0x7f27380a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:45.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.011+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f27440cf450 con 0x7f2754108130 2024-08-01T04:15:45.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.011+0000 7f2752a00640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27380a2c40 0x7f27380a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:45.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.012+0000 7f2752a00640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27380a2c40 0x7f27380a5100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2740031440 tx=0x7f27400315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:45.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.013+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2744021020 con 0x7f2754108130 2024-08-01T04:15:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.230+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2754062360 con 0x7f27380a2c40 2024-08-01T04:15:45.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.246+0000 7f274be00640 1 -- 172.21.15.3:0/237192576 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2754062360 con 0x7f27380a2c40 2024-08-01T04:15:45.247 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-01T04:15:45.247 INFO:teuthology.orchestra.run.smithi003.stdout:mds.all 12/12 53s ago 2m smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi163=c;smithi163=f;smithi163=i;smithi163=l;count:12 2024-08-01T04:15:45.248 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 53s ago 7m smithi003=x;smithi160=y;smithi163=z;count:3 2024-08-01T04:15:45.248 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 53s ago 8m smithi003:172.21.15.3=a;smithi160:172.21.15.160=b;smithi163:172.21.15.163=c;count:3 2024-08-01T04:15:45.248 INFO:teuthology.orchestra.run.smithi003.stdout:osd 12 53s ago - 2024-08-01T04:15:45.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27380a2c40 msgr2=0x7f27380a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:45.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27380a2c40 0x7f27380a5100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2740031440 tx=0x7f27400315b0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 msgr2=0x7f275419e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 0x7f275419e070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f274400d950 tx=0x7f274400de10 comp rx=0 tx=0).stop 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 shutdown_connections 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f27380a2c40 0x7f27380a5100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2754109930 0x7f27541a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754108f70 0x7f275419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 --2- 172.21.15.3:0/237192576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2754108130 0x7f275419e070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.249+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 >> 172.21.15.3:0/237192576 conn(0x7f2754076f30 msgr2=0x7f275410c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.250+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 shutdown_connections 2024-08-01T04:15:45.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:45.250+0000 7f2759200640 1 -- 172.21.15.3:0/237192576 wait complete. 2024-08-01T04:15:45.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:45 smithi003 ceph-mon[32262]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:45.598 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:45 smithi163 ceph-mon[37383]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:45.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:45 smithi160 ceph-mon[37224]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:15:46.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-01T04:15:46.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:46 smithi003 ceph-mon[32262]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:46.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:46 smithi160 ceph-mon[37224]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:46.775 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:46.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:46 smithi163 ceph-mon[37383]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:47.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 -- 172.21.15.3:0/174749450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28101770 msgr2=0x7f7b28101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 --2- 172.21.15.3:0/174749450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28101770 0x7f7b28101b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009be0 tx=0x7f7b1802a6d0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 -- 172.21.15.3:0/174749450 shutdown_connections 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 --2- 172.21.15.3:0/174749450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28103330 0x7f7b28109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 --2- 172.21.15.3:0/174749450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b28102970 0x7f7b28102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 --2- 172.21.15.3:0/174749450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28101770 0x7f7b28101b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 -- 172.21.15.3:0/174749450 >> 172.21.15.3:0/174749450 conn(0x7f7b280fcf20 msgr2=0x7f7b280ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 -- 172.21.15.3:0/174749450 shutdown_connections 2024-08-01T04:15:47.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.413+0000 7f7b2d800640 1 -- 172.21.15.3:0/174749450 wait complete. 2024-08-01T04:15:47.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.415+0000 7f7b2d800640 1 Processor -- start 2024-08-01T04:15:47.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.415+0000 7f7b2d800640 1 -- start start 2024-08-01T04:15:47.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.415+0000 7f7b2d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 0x7f7b28199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:47.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b2d800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b28102970 0x7f7b2819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b2d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28103330 0x7f7b281a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b2d800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2810d380 con 0x7f7b28101770 2024-08-01T04:15:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b2d800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2810d200 con 0x7f7b28102970 2024-08-01T04:15:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b2d800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2810d500 con 0x7f7b28103330 2024-08-01T04:15:47.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 0x7f7b28199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 0x7f7b28199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48542/0 (socket says 172.21.15.3:48542) 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 -- 172.21.15.3:0/2012241399 learned_addr learned my addr 172.21.15.3:0/2012241399 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b2ce00640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28103330 0x7f7b281a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27400640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b28102970 0x7f7b2819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 -- 172.21.15.3:0/2012241399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28103330 msgr2=0x7f7b281a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28103330 0x7f7b281a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 -- 172.21.15.3:0/2012241399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b28102970 msgr2=0x7f7b2819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b28102970 0x7f7b2819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.416+0000 7f7b27e00640 1 -- 172.21.15.3:0/2012241399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b281a18a0 con 0x7f7b28101770 2024-08-01T04:15:47.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.417+0000 7f7b27e00640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 0x7f7b28199c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7b18030960 tx=0x7f7b180305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:47.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.417+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18041020 con 0x7f7b28101770 2024-08-01T04:15:47.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.417+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b281a1b30 con 0x7f7b28101770 2024-08-01T04:15:47.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.417+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b18037040 con 0x7f7b28101770 2024-08-01T04:15:47.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.417+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18036070 con 0x7f7b28101770 2024-08-01T04:15:47.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.417+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281a2010 con 0x7f7b28101770 2024-08-01T04:15:47.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.418+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b281a21e0 con 0x7f7b28101770 2024-08-01T04:15:47.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.420+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7b18044020 con 0x7f7b28101770 2024-08-01T04:15:47.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.423+0000 7f7b24c00640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7b0c0a2c10 0x7f7b0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:47.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.423+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8269+0+0 (secure 0 0 0) 0x7f7b180efdd0 con 0x7f7b28101770 2024-08-01T04:15:47.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.423+0000 7f7b27400640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7b0c0a2c10 0x7f7b0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:47.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.424+0000 7f7b27400640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7b0c0a2c10 0x7f7b0c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b2819b190 tx=0x7f7b10009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:47.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.427+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7b1804b020 con 0x7f7b28101770 2024-08-01T04:15:47.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:47 smithi003 ceph-mon[32262]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.632+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7b28109860 con 0x7f7b0c0a2c10 2024-08-01T04:15:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.635+0000 7f7b24c00640 1 -- 172.21.15.3:0/2012241399 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7b28109860 con 0x7f7b0c0a2c10 2024-08-01T04:15:47.636 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2024-08-01T04:15:47.636 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2024-08-01T04:15:47.636 INFO:teuthology.orchestra.run.smithi003.stdout:smithi160 172.21.15.160 2024-08-01T04:15:47.636 INFO:teuthology.orchestra.run.smithi003.stdout:smithi163 172.21.15.163 2024-08-01T04:15:47.636 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.637+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7b0c0a2c10 msgr2=0x7f7b0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.637+0000 7f7b2d800640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7b0c0a2c10 0x7f7b0c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b2819b190 tx=0x7f7b10009260 comp rx=0 tx=0).stop 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.637+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 msgr2=0x7f7b28199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.637+0000 7f7b2d800640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 0x7f7b28199c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7b18030960 tx=0x7f7b180305b0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 shutdown_connections 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f7b0c0a2c10 0x7f7b0c0a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b28103330 0x7f7b281a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b28102970 0x7f7b2819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 --2- 172.21.15.3:0/2012241399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b28101770 0x7f7b28199c80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 >> 172.21.15.3:0/2012241399 conn(0x7f7b280fcf20 msgr2=0x7f7b28108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 shutdown_connections 2024-08-01T04:15:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:47.638+0000 7f7b2d800640 1 -- 172.21.15.3:0/2012241399 wait complete. 2024-08-01T04:15:47.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:47 smithi160 ceph-mon[37224]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:47.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:47 smithi163 ceph-mon[37383]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:15:48.628 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:48 smithi003 ceph-mon[32262]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:48.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-01T04:15:48.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:48 smithi160 ceph-mon[37224]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:48.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:48 smithi163 ceph-mon[37383]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:49.125 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:49.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:49 smithi003 ceph-mon[32262]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:49.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-08-01T04:15:49.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:49 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-08-01T04:15:49.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:49 smithi160 ceph-mon[37224]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:49.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-08-01T04:15:49.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:49 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-08-01T04:15:49.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:49 smithi163 ceph-mon[37383]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:15:49.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-08-01T04:15:49.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:49 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-08-01T04:15:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 -- 172.21.15.3:0/1635278403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 msgr2=0x7f1a4c10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 --2- 172.21.15.3:0/1635278403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c10bdc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009be0 tx=0x7f1a4402a6c0 comp rx=0 tx=0).stop 2024-08-01T04:15:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 -- 172.21.15.3:0/1635278403 shutdown_connections 2024-08-01T04:15:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 --2- 172.21.15.3:0/1635278403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c10bdc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 --2- 172.21.15.3:0/1635278403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c104b70 0x7f1a4c104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 --2- 172.21.15.3:0/1635278403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c103970 0x7f1a4c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 -- 172.21.15.3:0/1635278403 >> 172.21.15.3:0/1635278403 conn(0x7f1a4c0ff100 msgr2=0x7f1a4c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 -- 172.21.15.3:0/1635278403 shutdown_connections 2024-08-01T04:15:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.887+0000 7f1a4be00640 1 -- 172.21.15.3:0/1635278403 wait complete. 2024-08-01T04:15:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.888+0000 7f1a4be00640 1 Processor -- start 2024-08-01T04:15:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 -- start start 2024-08-01T04:15:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c103970 0x7f1a4c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c104b70 0x7f1a4c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c10f5d0 con 0x7f1a4c103970 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c10f450 con 0x7f1a4c104b70 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c10f750 con 0x7f1a4c105530 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c103970 0x7f1a4c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c104b70 0x7f1a4c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c104b70 0x7f1a4c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:44372/0 (socket says 172.21.15.3:44372) 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4a000640 1 -- 172.21.15.3:0/3666511818 learned_addr learned my addr 172.21.15.3:0/3666511818 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:49.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.889+0000 7f1a4aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c103970 0x7f1a4c199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48552/0 (socket says 172.21.15.3:48552) 2024-08-01T04:15:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.890+0000 7f1a4b400640 1 -- 172.21.15.3:0/3666511818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c104b70 msgr2=0x7f1a4c19a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:49.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.890+0000 7f1a4b400640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c104b70 0x7f1a4c19a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.890+0000 7f1a4b400640 1 -- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c103970 msgr2=0x7f1a4c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.890+0000 7f1a4b400640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c103970 0x7f1a4c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.890+0000 7f1a4b400640 1 -- 172.21.15.3:0/3666511818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c1a1800 con 0x7f1a4c105530 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.890+0000 7f1a4b400640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c1a1190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a44034040 tx=0x7f1a440305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.891+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44041020 con 0x7f1a4c105530 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.891+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a44030740 con 0x7f1a4c105530 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.891+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c1a1a90 con 0x7f1a4c105530 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.891+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44036040 con 0x7f1a4c105530 2024-08-01T04:15:49.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.891+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c1a1f90 con 0x7f1a4c105530 2024-08-01T04:15:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.892+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c10ad20 con 0x7f1a4c105530 2024-08-01T04:15:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.894+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1a44044020 con 0x7f1a4c105530 2024-08-01T04:15:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.896+0000 7f1a43400640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1a300a2c30 0x7f1a300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.896+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8327+0+0 (secure 0 0 0) 0x7f1a440efc50 con 0x7f1a4c105530 2024-08-01T04:15:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.896+0000 7f1a4aa00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1a300a2c30 0x7f1a300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.897+0000 7f1a4aa00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1a300a2c30 0x7f1a300a50f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c1049d0 tx=0x7f1a3c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:49.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:49.900+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1a4404b020 con 0x7f1a4c105530 2024-08-01T04:15:50.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.103+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 --> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1a4c062360 con 0x7f1a300a2c30 2024-08-01T04:15:50.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.107+0000 7f1a43400640 1 -- 172.21.15.3:0/3666511818 <== mgr.14148 v2:172.21.15.3:6800/1617846146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1a4c062360 con 0x7f1a300a2c30 2024-08-01T04:15:50.109 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-01T04:15:50.109 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-01T04:15:50.110 INFO:teuthology.orchestra.run.smithi003.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-01T04:15:50.110 INFO:teuthology.orchestra.run.smithi003.stdout:smithi163 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-01T04:15:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1a300a2c30 msgr2=0x7f1a300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1a300a2c30 0x7f1a300a50f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c1049d0 tx=0x7f1a3c005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 msgr2=0x7f1a4c1a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c1a1190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a44034040 tx=0x7f1a440305d0 comp rx=0 tx=0).stop 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 shutdown_connections 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f1a300a2c30 0x7f1a300a50f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a4c105530 0x7f1a4c1a1190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a4c104b70 0x7f1a4c19a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 --2- 172.21.15.3:0/3666511818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a4c103970 0x7f1a4c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.110+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 >> 172.21.15.3:0/3666511818 conn(0x7f1a4c0ff100 msgr2=0x7f1a4c107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.111+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 shutdown_connections 2024-08-01T04:15:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:50.111+0000 7f1a4be00640 1 -- 172.21.15.3:0/3666511818 wait complete. 2024-08-01T04:15:50.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-08-01T04:15:50.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:50 smithi160 ceph-mon[37224]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-08-01T04:15:50.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:50 smithi160 ceph-mon[37224]: osdmap e89: 12 total, 12 up, 12 in 2024-08-01T04:15:50.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:50 smithi160 ceph-mon[37224]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-08-01T04:15:50.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-08-01T04:15:50.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:50 smithi163 ceph-mon[37383]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-08-01T04:15:50.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:50 smithi163 ceph-mon[37383]: osdmap e89: 12 total, 12 up, 12 in 2024-08-01T04:15:50.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:50 smithi163 ceph-mon[37383]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-08-01T04:15:50.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-08-01T04:15:50.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:50 smithi003 ceph-mon[32262]: from='mgr.14148 172.21.15.3:0/2631558753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-08-01T04:15:50.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:50 smithi003 ceph-mon[32262]: osdmap e89: 12 total, 12 up, 12 in 2024-08-01T04:15:50.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:50 smithi003 ceph-mon[32262]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-08-01T04:15:51.075 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-01T04:15:51.087 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:15:51.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-01T04:15:51.581 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:51.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:51 smithi160 ceph-mon[37224]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:51.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:51 smithi160 ceph-mon[37224]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:15:51.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:51 smithi160 ceph-mon[37224]: osdmap e90: 12 total, 12 up, 12 in 2024-08-01T04:15:51.732 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:51 smithi003 ceph-mon[32262]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:51.732 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:51 smithi003 ceph-mon[32262]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:15:51.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:51 smithi003 ceph-mon[32262]: osdmap e90: 12 total, 12 up, 12 in 2024-08-01T04:15:51.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:51 smithi163 ceph-mon[37383]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:15:51.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:51 smithi163 ceph-mon[37383]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:15:51.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:51 smithi163 ceph-mon[37383]: osdmap e90: 12 total, 12 up, 12 in 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 -- 172.21.15.3:0/3672103824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528105530 msgr2=0x7f352810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 --2- 172.21.15.3:0/3672103824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528105530 0x7f352810bdc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3520009ca0 tx=0x7f352002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 -- 172.21.15.3:0/3672103824 shutdown_connections 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 --2- 172.21.15.3:0/3672103824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528105530 0x7f352810bdc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 --2- 172.21.15.3:0/3672103824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528104b70 0x7f3528104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 --2- 172.21.15.3:0/3672103824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3528103970 0x7f3528103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 -- 172.21.15.3:0/3672103824 >> 172.21.15.3:0/3672103824 conn(0x7f35280ff100 msgr2=0x7f3528101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 -- 172.21.15.3:0/3672103824 shutdown_connections 2024-08-01T04:15:52.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.268+0000 7f3527200640 1 -- 172.21.15.3:0/3672103824 wait complete. 2024-08-01T04:15:52.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.270+0000 7f3527200640 1 Processor -- start 2024-08-01T04:15:52.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.270+0000 7f3527200640 1 -- start start 2024-08-01T04:15:52.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3527200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528103970 0x7f3528199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:52.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3527200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 0x7f352819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:52.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3527200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3528105530 0x7f35281a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:52.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3527200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f352810f5f0 con 0x7f3528104b70 2024-08-01T04:15:52.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3527200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f352810f470 con 0x7f3528105530 2024-08-01T04:15:52.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3527200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f352810f770 con 0x7f3528103970 2024-08-01T04:15:52.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f351fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 0x7f352819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:52.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f351fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 0x7f352819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55580/0 (socket says 172.21.15.3:55580) 2024-08-01T04:15:52.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f351fe00640 1 -- 172.21.15.3:0/96018936 learned_addr learned my addr 172.21.15.3:0/96018936 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:52.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3525e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528103970 0x7f3528199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:52.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.271+0000 7f3526800640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3528105530 0x7f35281a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351fe00640 1 -- 172.21.15.3:0/96018936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528103970 msgr2=0x7f3528199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351fe00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528103970 0x7f3528199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351fe00640 1 -- 172.21.15.3:0/96018936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3528105530 msgr2=0x7f35281a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351fe00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3528105530 0x7f35281a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351fe00640 1 -- 172.21.15.3:0/96018936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35281a1810 con 0x7f3528104b70 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f3525e00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528103970 0x7f3528199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351fe00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 0x7f352819a120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f351000e9f0 tx=0x7f351000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.272+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351000cd80 con 0x7f3528104b70 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.273+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3510005500 con 0x7f3528104b70 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.273+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35281a1b00 con 0x7f3528104b70 2024-08-01T04:15:52.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.273+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510010690 con 0x7f3528104b70 2024-08-01T04:15:52.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.273+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281a1ff0 con 0x7f3528104b70 2024-08-01T04:15:52.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.274+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352810ad20 con 0x7f3528104b70 2024-08-01T04:15:52.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.276+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f35100046c0 con 0x7f3528104b70 2024-08-01T04:15:52.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.278+0000 7f351ea00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f35080a2bb0 0x7f35080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:52.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.278+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f35100cf4e0 con 0x7f3528104b70 2024-08-01T04:15:52.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.278+0000 7f3525e00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f35080a2bb0 0x7f35080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:52.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.279+0000 7f3525e00640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f35080a2bb0 0x7f35080a5070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f35281049d0 tx=0x7f3518005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:52.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.282+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f351009c380 con 0x7f3528104b70 2024-08-01T04:15:52.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.515+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3528062b20 con 0x7f3528104b70 2024-08-01T04:15:52.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.516+0000 7f351ea00640 1 -- 172.21.15.3:0/96018936 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3085 (secure 0 0 0) 0x7f351009bc80 con 0x7f3528104b70 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:e25 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:btime 2024-08-01T04:14:12:627814+0000 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:legacy client fscid: 1 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem 'cephfs' (1) 2024-08-01T04:15:52.519 INFO:teuthology.orchestra.run.smithi003.stdout:fs_name cephfs 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 25 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-08-01T04:13:55.770404+0000 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-08-01T04:14:11.859614+0000 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:tableserver 0 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:root 0 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:session_timeout 300 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:session_autoclose 300 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:max_file_size 1099511627776 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:max_xattr_size 65536 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:required_client_features {} 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure 0 2024-08-01T04:15:52.520 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure_osd_epoch 0 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:max_mds 3 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:in 0,1,2 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:up {0=24548,1=24520,2=24506} 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:failed 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:damaged 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:stopped 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:data_pools [3] 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:metadata_pool 2 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:inline_data disabled 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:balancer 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:bal_rank_mask -1 2024-08-01T04:15:52.521 INFO:teuthology.orchestra.run.smithi003.stdout:standby_count_wanted 0 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:qdb_cluster leader: 24548 members: 24548,24520,24506 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b{0:24548} state up:active seq 5 addr [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.i{0:24502} state up:standby-replay seq 1 addr [v2:172.21.15.163:6838/4057809310,v1:172.21.15.163:6839/4057809310] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a{1:24520} state up:active seq 8 addr [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c{1:24487} state up:standby-replay seq 1 addr [v2:172.21.15.163:6836/1698078935,v1:172.21.15.163:6837/1698078935] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.h{2:24506} state up:active seq 11 addr [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.e{2:24485} state up:standby-replay seq 1 addr [v2:172.21.15.160:6834/3179303046,v1:172.21.15.160:6835/3179303046] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:Standby daemons: 2024-08-01T04:15:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.g{-1:14406} state up:standby seq 1 addr [v2:172.21.15.3:6838/367445798,v1:172.21.15.3:6839/367445798] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.l{-1:24433} state up:standby seq 1 addr [v2:172.21.15.163:6832/4230398308,v1:172.21.15.163:6833/4230398308] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.k{-1:24455} state up:standby seq 1 addr [v2:172.21.15.160:6832/3012672686,v1:172.21.15.160:6833/3012672686] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.j{-1:24467} state up:standby seq 1 addr [v2:172.21.15.3:6834/3318532700,v1:172.21.15.3:6835/3318532700] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.f{-1:24475} state up:standby seq 1 addr [v2:172.21.15.163:6834/2143574012,v1:172.21.15.163:6835/2143574012] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d{-1:24479} state up:standby seq 1 addr [v2:172.21.15.3:6836/258713238,v1:172.21.15.3:6837/258713238] compat {c=[1],r=[1],i=[1fff]}] 2024-08-01T04:15:52.523 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:15:52.524 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:52 smithi003 ceph-mon[32262]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f35080a2bb0 msgr2=0x7f35080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f35080a2bb0 0x7f35080a5070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f35281049d0 tx=0x7f3518005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 msgr2=0x7f352819a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 0x7f352819a120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f351000e9f0 tx=0x7f351000edf0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 shutdown_connections 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f35080a2bb0 0x7f35080a5070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3528105530 0x7f35281a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3528104b70 0x7f352819a120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 --2- 172.21.15.3:0/96018936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3528103970 0x7f3528199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 >> 172.21.15.3:0/96018936 conn(0x7f35280ff100 msgr2=0x7f3528107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.520+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 shutdown_connections 2024-08-01T04:15:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:52.521+0000 7f3527200640 1 -- 172.21.15.3:0/96018936 wait complete. 2024-08-01T04:15:52.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:52 smithi163 ceph-mon[37383]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-08-01T04:15:52.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:52 smithi160 ceph-mon[37224]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-08-01T04:15:53.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-01T04:15:53.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:53 smithi163 ceph-mon[37383]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:53.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:53 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/96018936' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-01T04:15:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:53 smithi003 ceph-mon[32262]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:53 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/96018936' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-01T04:15:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:53 smithi160 ceph-mon[37224]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:53 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/96018936' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-01T04:15:54.010 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:54.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 -- 172.21.15.3:0/1205468346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464104b40 msgr2=0x7fd464106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:54.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 --2- 172.21.15.3:0/1205468346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464104b40 0x7fd464106f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009ca0 tx=0x7fd44c02a8c0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 -- 172.21.15.3:0/1205468346 shutdown_connections 2024-08-01T04:15:54.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 --2- 172.21.15.3:0/1205468346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd464107470 0x7fd4641098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 --2- 172.21.15.3:0/1205468346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464104b40 0x7fd464106f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 --2- 172.21.15.3:0/1205468346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464102210 0x7fd464104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 -- 172.21.15.3:0/1205468346 >> 172.21.15.3:0/1205468346 conn(0x7fd4640fbe10 msgr2=0x7fd4640fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 -- 172.21.15.3:0/1205468346 shutdown_connections 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.694+0000 7fd469c00640 1 -- 172.21.15.3:0/1205468346 wait complete. 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.695+0000 7fd469c00640 1 Processor -- start 2024-08-01T04:15:54.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.695+0000 7fd469c00640 1 -- start start 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 0x7fd464199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464107470 0x7fd46419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd46419f480 0x7fd4641a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd46410d460 con 0x7fd464102210 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd46410d2e0 con 0x7fd46419f480 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd46410d5e0 con 0x7fd464107470 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd463e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 0x7fd464199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd45b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464107470 0x7fd46419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd463e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 0x7fd464199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55598/0 (socket says 172.21.15.3:55598) 2024-08-01T04:15:54.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd463e00640 1 -- 172.21.15.3:0/2571004333 learned_addr learned my addr 172.21.15.3:0/2571004333 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.696+0000 7fd469200640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd46419f480 0x7fd4641a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd45b400640 1 -- 172.21.15.3:0/2571004333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd46419f480 msgr2=0x7fd4641a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd45b400640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd46419f480 0x7fd4641a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd45b400640 1 -- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 msgr2=0x7fd464199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd45b400640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 0x7fd464199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd45b400640 1 -- 172.21.15.3:0/2571004333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641a1ee0 con 0x7fd464107470 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd463e00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 0x7fd464199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd45b400640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464107470 0x7fd46419a410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd46406a3a0 tx=0x7fd44c0319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c003b70 con 0x7fd464107470 2024-08-01T04:15:54.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4641a2170 con 0x7fd464107470 2024-08-01T04:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd44c003d10 con 0x7fd464107470 2024-08-01T04:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c02fbd0 con 0x7fd464107470 2024-08-01T04:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.697+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a2680 con 0x7fd464107470 2024-08-01T04:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.699+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd464108fc0 con 0x7fd464107470 2024-08-01T04:15:54.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.701+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd44c02fd70 con 0x7fd464107470 2024-08-01T04:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.702+0000 7fd461600640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd4480a2c30 0x7fd4480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.702+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7fd44c0f03c0 con 0x7fd464107470 2024-08-01T04:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.702+0000 7fd463e00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd4480a2c30 0x7fd4480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.703+0000 7fd463e00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd4480a2c30 0x7fd4480a50f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd454011500 tx=0x7fd454005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:54.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.704+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd44c0bd260 con 0x7fd464107470 2024-08-01T04:15:54.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.900+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd46402d5e0 con 0x7fd464107470 2024-08-01T04:15:54.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.901+0000 7fd461600640 1 -- 172.21.15.3:0/2571004333 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5370 (secure 0 0 0) 0x7fd44c0bcb60 con 0x7fd464107470 2024-08-01T04:15:54.906 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 90 2024-08-01T04:15:54.906 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:15:54.906 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-08-01T04:05:46.214560+0000 2024-08-01T04:15:54.906 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-08-01T04:15:50.375521+0000 2024-08-01T04:15:54.906 INFO:teuthology.orchestra.run.smithi003.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:crush_version 26 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:full_ratio 0.95 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:backfillfull_ratio 0.9 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:nearfull_ratio 0.85 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:require_min_compat_client luminous 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:min_compat_client luminous 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:require_osd_release squid 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:stretch_mode_enabled false 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:max_osd 12 2024-08-01T04:15:54.907 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6802/1021833457,v1:172.21.15.3:6803/1021833457] [v2:172.21.15.3:6804/1021833457,v1:172.21.15.3:6805/1021833457] exists,up f0fdfd41-e466-4275-8212-709184cca9a6 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6800/1858231238,v1:172.21.15.160:6801/1858231238] [v2:172.21.15.160:6802/1858231238,v1:172.21.15.160:6803/1858231238] exists,up f9b159fc-e987-4b22-b63e-910d2fc377e3 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6800/2416358231,v1:172.21.15.163:6801/2416358231] [v2:172.21.15.163:6802/2416358231,v1:172.21.15.163:6803/2416358231] exists,up 90f0ad0d-8599-4670-868b-e0b0a593aa0d 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6810/1408863124,v1:172.21.15.3:6811/1408863124] [v2:172.21.15.3:6812/1408863124,v1:172.21.15.3:6813/1408863124] exists,up 97d21a34-b1bd-4f38-ac45-28f4f9360b54 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6808/1218975759,v1:172.21.15.160:6809/1218975759] [v2:172.21.15.160:6810/1218975759,v1:172.21.15.160:6811/1218975759] exists,up 7838d1fc-3355-4659-b61b-5fa5717d70b6 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6808/1207930650,v1:172.21.15.163:6809/1207930650] [v2:172.21.15.163:6810/1207930650,v1:172.21.15.163:6811/1207930650] exists,up 2f0ce098-7164-49a0-9d74-1c6afc6a32c3 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 up in weight 1 up_from 46 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6818/885876521,v1:172.21.15.3:6819/885876521] [v2:172.21.15.3:6820/885876521,v1:172.21.15.3:6821/885876521] exists,up b3de683d-970d-4e25-9574-48cf63d401ae 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 up in weight 1 up_from 51 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6816/3141457315,v1:172.21.15.160:6817/3141457315] [v2:172.21.15.160:6818/3141457315,v1:172.21.15.160:6819/3141457315] exists,up e2878577-9071-4b13-92ed-0cace6099cfd 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 up in weight 1 up_from 58 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6816/2678360734,v1:172.21.15.163:6817/2678360734] [v2:172.21.15.163:6818/2678360734,v1:172.21.15.163:6819/2678360734] exists,up c865f68f-ddd2-4332-bbfc-26b68a4b85b8 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 up in weight 1 up_from 66 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6826/2204833975,v1:172.21.15.3:6827/2204833975] [v2:172.21.15.3:6828/2204833975,v1:172.21.15.3:6829/2204833975] exists,up f91a1874-e1c3-40ec-b6de-55b00c8d40e5 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 up in weight 1 up_from 71 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6824/1295991046,v1:172.21.15.160:6825/1295991046] [v2:172.21.15.160:6826/1295991046,v1:172.21.15.160:6827/1295991046] exists,up 64b32252-f40b-450c-bacc-c37ac7660b3e 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 up in weight 1 up_from 78 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.163:6824/2977436589,v1:172.21.15.163:6825/2977436589] [v2:172.21.15.163:6826/2977436589,v1:172.21.15.163:6827/2977436589] exists,up e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.a [2,11] 2024-08-01T04:15:54.908 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.e [2,1] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1a [8,4] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1d [8,11] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.6 [4,7] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.10 [5,9] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.14 [4,6] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.1d [4,8] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.22 [3,9] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.28 [5,8] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.2a [3,6] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.3c [4,9] 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/2156560379 expires 2024-08-02T04:06:48.053573+0000 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/2156560379 expires 2024-08-02T04:06:48.053573+0000 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/4012679146 expires 2024-08-02T04:06:48.053573+0000 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2792822839 expires 2024-08-02T04:06:48.053573+0000 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/1530825959 expires 2024-08-02T04:06:17.351564+0000 2024-08-01T04:15:54.909 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1368241138 expires 2024-08-02T04:06:48.053573+0000 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2048633599 expires 2024-08-02T04:06:17.351564+0000 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1579863007 expires 2024-08-02T04:06:17.351564+0000 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/1530825959 expires 2024-08-02T04:06:17.351564+0000 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/320273213 expires 2024-08-02T04:06:17.351564+0000 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd4480a2c30 msgr2=0x7fd4480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd4480a2c30 0x7fd4480a50f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd454011500 tx=0x7fd454005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:54.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464107470 msgr2=0x7fd46419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464107470 0x7fd46419a410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd46406a3a0 tx=0x7fd44c0319f0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 shutdown_connections 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fd4480a2c30 0x7fd4480a50f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd46419f480 0x7fd4641a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd464107470 0x7fd46419a410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.904+0000 7fd469c00640 1 --2- 172.21.15.3:0/2571004333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd464102210 0x7fd464199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.905+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 >> 172.21.15.3:0/2571004333 conn(0x7fd4640fbe10 msgr2=0x7fd464101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.905+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 shutdown_connections 2024-08-01T04:15:54.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:54.905+0000 7fd469c00640 1 -- 172.21.15.3:0/2571004333 wait complete. 2024-08-01T04:15:55.673 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:55 smithi003 ceph-mon[32262]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:55.674 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:55 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2571004333' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-01T04:15:55.741 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-01T04:15:55.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:15:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:55 smithi163 ceph-mon[37383]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:55 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2571004333' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-01T04:15:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:55 smithi160 ceph-mon[37224]: pgmap v353: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-08-01T04:15:55.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:55 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2571004333' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-01T04:15:56.240 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:56.672 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:56 smithi003 ceph-mon[32262]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-08-01T04:15:56.672 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:56 smithi003 ceph-mon[32262]: Cluster is now healthy 2024-08-01T04:15:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:56 smithi163 ceph-mon[37383]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-08-01T04:15:56.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:56 smithi163 ceph-mon[37383]: Cluster is now healthy 2024-08-01T04:15:56.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:56 smithi160 ceph-mon[37224]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-08-01T04:15:56.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:56 smithi160 ceph-mon[37224]: Cluster is now healthy 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 -- 172.21.15.3:0/4065898251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0109b60 msgr2=0x7f48a0109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 --2- 172.21.15.3:0/4065898251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0109b60 0x7f48a0109f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4890009900 tx=0x7f489002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 -- 172.21.15.3:0/4065898251 shutdown_connections 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 --2- 172.21.15.3:0/4065898251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 0x7f48a01109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 --2- 172.21.15.3:0/4065898251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0103ad0 0x7f48a0103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 --2- 172.21.15.3:0/4065898251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0109b60 0x7f48a0109f40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 -- 172.21.15.3:0/4065898251 >> 172.21.15.3:0/4065898251 conn(0x7f48a00ff750 msgr2=0x7f48a0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.967+0000 7f48a5800640 1 -- 172.21.15.3:0/4065898251 shutdown_connections 2024-08-01T04:15:56.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.968+0000 7f48a5800640 1 -- 172.21.15.3:0/4065898251 wait complete. 2024-08-01T04:15:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.969+0000 7f48a5800640 1 Processor -- start 2024-08-01T04:15:56.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 -- start start 2024-08-01T04:15:56.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0103ad0 0x7f48a019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 0x7f48a019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 0x7f48a019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0113f20 con 0x7f48a0109b60 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0113da0 con 0x7f48a0103ad0 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.970+0000 7f48a5800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f48a01140a0 con 0x7f48a0104470 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 0x7f48a019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0103ad0 0x7f48a019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 0x7f48a019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55622/0 (socket says 172.21.15.3:55622) 2024-08-01T04:15:56.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 -- 172.21.15.3:0/1702379105 learned_addr learned my addr 172.21.15.3:0/1702379105 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f4896000640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 0x7f48a019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 -- 172.21.15.3:0/1702379105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 msgr2=0x7f48a019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 0x7f48a019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 -- 172.21.15.3:0/1702379105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0103ad0 msgr2=0x7f48a019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0103ad0 0x7f48a019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f489f400640 1 -- 172.21.15.3:0/1702379105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a019f350 con 0x7f48a0109b60 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.971+0000 7f4896000640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 0x7f48a019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:15:56.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.972+0000 7f489f400640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 0x7f48a019ec70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f489800eb10 tx=0x7f489800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.972+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898004b20 con 0x7f48a0109b60 2024-08-01T04:15:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.972+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4898004cc0 con 0x7f48a0109b60 2024-08-01T04:15:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.972+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a019f640 con 0x7f48a0109b60 2024-08-01T04:15:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.972+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898010760 con 0x7f48a0109b60 2024-08-01T04:15:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.972+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a01a6f20 con 0x7f48a0109b60 2024-08-01T04:15:56.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.974+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a0103f30 con 0x7f48a0109b60 2024-08-01T04:15:56.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.975+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f4898005020 con 0x7f48a0109b60 2024-08-01T04:15:56.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.977+0000 7f4897e00640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48800a2c00 0x7f48800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:56.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.977+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f48980cf780 con 0x7f48a0109b60 2024-08-01T04:15:56.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.977+0000 7f489ea00640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48800a2c00 0x7f48800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:56.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.978+0000 7f489ea00640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48800a2c00 0x7f48800a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f4890005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:56.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:56.981+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f489809c620 con 0x7f48a0109b60 2024-08-01T04:15:57.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.214+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48a002d5e0 con 0x7f48a0109b60 2024-08-01T04:15:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.216+0000 7f4897e00640 1 -- 172.21.15.3:0/1702379105 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7f489809bf20 con 0x7f48a0109b60 2024-08-01T04:15:57.218 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:57.218 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-08-01T04:14:12:627814+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:14:11.859614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:15:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48800a2c00 msgr2=0x7f48800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48800a2c00 0x7f48800a50c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f4890005b50 comp rx=0 tx=0).stop 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 msgr2=0x7f48a019ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 0x7f48a019ec70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f489800eb10 tx=0x7f489800c2d0 comp rx=0 tx=0).stop 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 shutdown_connections 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f48800a2c00 0x7f48800a50c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a0109b60 0x7f48a019ec70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:57.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a0104470 0x7f48a019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 --2- 172.21.15.3:0/1702379105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48a0103ad0 0x7f48a019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.219+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 >> 172.21.15.3:0/1702379105 conn(0x7f48a00ff750 msgr2=0x7f48a00fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.220+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 shutdown_connections 2024-08-01T04:15:57.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:57.220+0000 7f48a5800640 1 -- 172.21.15.3:0/1702379105 wait complete. 2024-08-01T04:15:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:57 smithi163 ceph-mon[37383]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-01T04:15:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:57 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1702379105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:57 smithi003 ceph-mon[32262]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-01T04:15:57.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:57 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1702379105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:57.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:57 smithi160 ceph-mon[37224]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-01T04:15:57.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:57 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1702379105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:15:58.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:15:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.279+0000 7fbd99000640 1 -- 172.21.15.3:0/4176965634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9410df90 msgr2=0x7fbd94110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.279+0000 7fbd99000640 1 --2- 172.21.15.3:0/4176965634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9410df90 0x7fbd94110380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009900 tx=0x7fbd8c02a480 comp rx=0 tx=0).stop 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 -- 172.21.15.3:0/4176965634 shutdown_connections 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 --2- 172.21.15.3:0/4176965634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd9410df90 0x7fbd94110380 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 --2- 172.21.15.3:0/4176965634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd94074c10 0x7fbd9410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 --2- 172.21.15.3:0/4176965634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd940742f0 0x7fbd940746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 -- 172.21.15.3:0/4176965634 >> 172.21.15.3:0/4176965634 conn(0x7fbd940fd470 msgr2=0x7fbd940ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 -- 172.21.15.3:0/4176965634 shutdown_connections 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.280+0000 7fbd99000640 1 -- 172.21.15.3:0/4176965634 wait complete. 2024-08-01T04:15:59.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 Processor -- start 2024-08-01T04:15:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 -- start start 2024-08-01T04:15:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 0x7fbd9419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd94074c10 0x7fbd9419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:59.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410df90 0x7fbd9419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94113d30 con 0x7fbd940742f0 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94113bb0 con 0x7fbd9410df90 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.281+0000 7fbd99000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94113eb0 con 0x7fbd94074c10 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 0x7fbd9419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 0x7fbd9419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55640/0 (socket says 172.21.15.3:55640) 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 -- 172.21.15.3:0/1403874372 learned_addr learned my addr 172.21.15.3:0/1403874372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd92a00640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd94074c10 0x7fbd9419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93e00640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410df90 0x7fbd9419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 -- 172.21.15.3:0/1403874372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd94074c10 msgr2=0x7fbd9419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd94074c10 0x7fbd9419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 -- 172.21.15.3:0/1403874372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410df90 msgr2=0x7fbd9419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:59.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410df90 0x7fbd9419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.282+0000 7fbd93400640 1 -- 172.21.15.3:0/1403874372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9419f160 con 0x7fbd940742f0 2024-08-01T04:15:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.283+0000 7fbd93400640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 0x7fbd9419a1b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400eb90 tx=0x7fbd8400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.283+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004d20 con 0x7fbd940742f0 2024-08-01T04:15:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.283+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd8400ee90 con 0x7fbd940742f0 2024-08-01T04:15:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.283+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9419f450 con 0x7fbd940742f0 2024-08-01T04:15:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.283+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941a6d30 con 0x7fbd940742f0 2024-08-01T04:15:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.284+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84010830 con 0x7fbd940742f0 2024-08-01T04:15:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.285+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd70005290 con 0x7fbd940742f0 2024-08-01T04:15:59.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.287+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbd84005020 con 0x7fbd940742f0 2024-08-01T04:15:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.288+0000 7fbd8be00640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:15:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.288+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7fbd840cf650 con 0x7fbd940742f0 2024-08-01T04:15:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.289+0000 7fbd92a00640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:15:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.289+0000 7fbd92a00640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd780a2b80 0x7fbd780a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd9410f320 tx=0x7fbd7c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:15:59.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.293+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbd8409c4f0 con 0x7fbd940742f0 2024-08-01T04:15:59.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.532+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd70005580 con 0x7fbd940742f0 2024-08-01T04:15:59.535 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:15:59 smithi003 ceph-mon[32262]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 693 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-01T04:15:59.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.534+0000 7fbd8be00640 1 -- 172.21.15.3:0/1403874372 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11908 (secure 0 0 0) 0x7fbd8409bdf0 con 0x7fbd940742f0 2024-08-01T04:15:59.537 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:15:59.537 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-08-01T04:14:12:627814+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:14:11.859614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:15:59.537 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-08-01T04:15:59.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd780a2b80 msgr2=0x7fbd780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:59.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd780a2b80 0x7fbd780a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbd9410f320 tx=0x7fbd7c005f90 comp rx=0 tx=0).stop 2024-08-01T04:15:59.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 msgr2=0x7fbd9419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:15:59.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 0x7fbd9419a1b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400eb90 tx=0x7fbd8400c6a0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 shutdown_connections 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410df90 0x7fbd9419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd94074c10 0x7fbd9419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 --2- 172.21.15.3:0/1403874372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd940742f0 0x7fbd9419a1b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.537+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 >> 172.21.15.3:0/1403874372 conn(0x7fbd940fd470 msgr2=0x7fbd940fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.538+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 shutdown_connections 2024-08-01T04:15:59.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:15:59.538+0000 7fbd99000640 1 -- 172.21.15.3:0/1403874372 wait complete. 2024-08-01T04:15:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:15:59 smithi163 ceph-mon[37383]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 693 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-01T04:15:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:15:59 smithi160 ceph-mon[37224]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 693 B/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-08-01T04:16:00.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:16:00.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:00 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1403874372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:00.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:00 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1403874372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:00.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:00 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1403874372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:01.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.625+0000 7f21f3600640 1 -- 172.21.15.3:0/3042067457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 msgr2=0x7f21ec10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:01.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.625+0000 7f21f3600640 1 --2- 172.21.15.3:0/3042067457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 0x7f21ec10cb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009900 tx=0x7f21d402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 -- 172.21.15.3:0/3042067457 shutdown_connections 2024-08-01T04:16:01.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 --2- 172.21.15.3:0/3042067457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec10d260 0x7f21ec114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 --2- 172.21.15.3:0/3042067457 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 0x7f21ec10cb30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 --2- 172.21.15.3:0/3042067457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec108180 0x7f21ec108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 -- 172.21.15.3:0/3042067457 >> 172.21.15.3:0/3042067457 conn(0x7f21ec0778d0 msgr2=0x7f21ec079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:01.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 -- 172.21.15.3:0/3042067457 shutdown_connections 2024-08-01T04:16:01.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.626+0000 7f21f3600640 1 -- 172.21.15.3:0/3042067457 wait complete. 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.627+0000 7f21f3600640 1 Processor -- start 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.627+0000 7f21f3600640 1 -- start start 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.627+0000 7f21f3600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 0x7f21ec1a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.627+0000 7f21f3600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 0x7f21ec1a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.627+0000 7f21f3600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec10d260 0x7f21ec19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f3600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1182d0 con 0x7f21ec108b30 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f3600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec118150 con 0x7f21ec108180 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f3600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec118450 con 0x7f21ec10d260 2024-08-01T04:16:01.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21e3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 0x7f21ec1a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:01.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 0x7f21ec1a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:01.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 0x7f21ec1a4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:35060/0 (socket says 172.21.15.3:35060) 2024-08-01T04:16:01.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 -- 172.21.15.3:0/2020068750 learned_addr learned my addr 172.21.15.3:0/2020068750 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:01.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f1600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec10d260 0x7f21ec19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 -- 172.21.15.3:0/2020068750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec10d260 msgr2=0x7f21ec19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec10d260 0x7f21ec19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 -- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 msgr2=0x7f21ec1a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 0x7f21ec1a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.628+0000 7f21f0c00640 1 -- 172.21.15.3:0/2020068750 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec19ee90 con 0x7f21ec108180 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.629+0000 7f21f0c00640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 0x7f21ec1a4500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00e9f0 tx=0x7f21dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.629+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00cd80 con 0x7f21ec108180 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.629+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec19f180 con 0x7f21ec108180 2024-08-01T04:16:01.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.629+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21dc005500 con 0x7f21ec108180 2024-08-01T04:16:01.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.629+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc010690 con 0x7f21ec108180 2024-08-01T04:16:01.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.629+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1ab4f0 con 0x7f21ec108180 2024-08-01T04:16:01.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.630+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec109610 con 0x7f21ec108180 2024-08-01T04:16:01.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.632+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f21dc0046c0 con 0x7f21ec108180 2024-08-01T04:16:01.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.634+0000 7f21ea000640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f21d00a2c00 0x7f21d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:01.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.634+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f21dc0cf480 con 0x7f21ec108180 2024-08-01T04:16:01.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.634+0000 7f21e3e00640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f21d00a2c00 0x7f21d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:01.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.635+0000 7f21e3e00640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f21d00a2c00 0x7f21d00a50c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d4005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:01.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.638+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f21dc09c320 con 0x7f21ec108180 2024-08-01T04:16:01.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:01 smithi163 ceph-mon[37383]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:01.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:01 smithi003 ceph-mon[32262]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:01.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.833+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21ec1104a0 con 0x7f21ec108180 2024-08-01T04:16:01.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.835+0000 7f21ea000640 1 -- 172.21.15.3:0/2020068750 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22807 (secure 0 0 0) 0x7f21dc086070 con 0x7f21ec108180 2024-08-01T04:16:01.837 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:01.838 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:15:50.375521+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:16:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f21d00a2c00 msgr2=0x7f21d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f21d00a2c00 0x7f21d00a50c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21d4000c00 tx=0x7f21d4005b50 comp rx=0 tx=0).stop 2024-08-01T04:16:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 msgr2=0x7f21ec1a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 0x7f21ec1a4500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00e9f0 tx=0x7f21dc00edf0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 shutdown_connections 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f21d00a2c00 0x7f21d00a50c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21ec10d260 0x7f21ec19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108b30 0x7f21ec1a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 --2- 172.21.15.3:0/2020068750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21ec108180 0x7f21ec1a4500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 >> 172.21.15.3:0/2020068750 conn(0x7f21ec0778d0 msgr2=0x7f21ec078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 shutdown_connections 2024-08-01T04:16:01.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:01.838+0000 7f21f3600640 1 -- 172.21.15.3:0/2020068750 wait complete. 2024-08-01T04:16:01.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:01 smithi160 ceph-mon[37224]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:02.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-01T04:13:55.770404+0000', 'modified': '2024-08-01T04:14:11.859614+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24520, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/3179303046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 3179303046}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 3179303046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/1698078935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1698078935}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1698078935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6839/4057809310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 4057809310}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 4057809310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.160:6837/3127569351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 3127569351}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 3127569351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.3:6841/2435488646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 2435488646}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 2435488646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.160:6839/1741438883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 1741438883}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 1741438883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24548, 'qdb_cluster': [24548, 24520, 24506]} 2024-08-01T04:16:02.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-01T04:16:02.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.h status 2024-08-01T04:16:02.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:02 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2020068750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:02.855 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:02 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2020068750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:02.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:02 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2020068750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:03.210 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:03.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:03 smithi003 ceph-mon[32262]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 518 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:03.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:03 smithi163 ceph-mon[37383]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 518 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 -- 172.21.15.3:0/514025151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 msgr2=0x7f114c110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 --2- 172.21.15.3:0/514025151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c110380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1144009ca0 tx=0x7f114402a940 comp rx=0 tx=0).stop 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 -- 172.21.15.3:0/514025151 shutdown_connections 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 --2- 172.21.15.3:0/514025151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c110380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 --2- 172.21.15.3:0/514025151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074c70 0x7f114c10d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 --2- 172.21.15.3:0/514025151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f114c074350 0x7f114c074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 -- 172.21.15.3:0/514025151 >> 172.21.15.3:0/514025151 conn(0x7f114c0fd410 msgr2=0x7f114c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.852+0000 7f1151e00640 1 -- 172.21.15.3:0/514025151 shutdown_connections 2024-08-01T04:16:03.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.853+0000 7f1151e00640 1 -- 172.21.15.3:0/514025151 wait complete. 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.854+0000 7f1151e00640 1 Processor -- start 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 -- start start 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074350 0x7f114c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f114c074c70 0x7f114c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f114c113d00 con 0x7f114c10df30 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f114c113b80 con 0x7f114c074350 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f114c113e80 con 0x7f114c074c70 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1150a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074350 0x7f114c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51762/0 (socket says 172.21.15.3:51762) 2024-08-01T04:16:03.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f114be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f114c074c70 0x7f114c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.855+0000 7f1151400640 1 -- 172.21.15.3:0/604843903 learned_addr learned my addr 172.21.15.3:0/604843903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1151400640 1 -- 172.21.15.3:0/604843903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f114c074c70 msgr2=0x7f114c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1151400640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f114c074c70 0x7f114c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1151400640 1 -- 172.21.15.3:0/604843903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074350 msgr2=0x7f114c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1151400640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074350 0x7f114c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1151400640 1 -- 172.21.15.3:0/604843903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c19f1b0 con 0x7f114c10df30 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1150a00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074350 0x7f114c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.856+0000 7f1151400640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c19ead0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1144005e30 tx=0x7f114402f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.857+0000 7f1149600640 1 -- 172.21.15.3:0/604843903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144034040 con 0x7f114c10df30 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.857+0000 7f1149600640 1 -- 172.21.15.3:0/604843903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114402fe10 con 0x7f114c10df30 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.857+0000 7f1149600640 1 -- 172.21.15.3:0/604843903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11440045d0 con 0x7f114c10df30 2024-08-01T04:16:03.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.857+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c19f440 con 0x7f114c10df30 2024-08-01T04:16:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.857+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c19f8d0 con 0x7f114c10df30 2024-08-01T04:16:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.860+0000 7f1149600640 1 -- 172.21.15.3:0/604843903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1144037020 con 0x7f114c10df30 2024-08-01T04:16:03.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 Processor -- start 2024-08-01T04:16:03.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 -- start start 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc22920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec8db88940 con 0x55ec8dc2d280 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ec8db209a0 con 0x55ec8dc23a50 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1155e7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec8db84900 con 0x55ec8dc224e0 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1150a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.861+0000 7f1150a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc22920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:33044/0 (socket says 172.21.15.3:33044) 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 -- 172.21.15.3:0/1176317527 learned_addr learned my addr 172.21.15.3:0/1176317527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f114be00640 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f114be00640 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 -- 172.21.15.3:0/1176317527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 msgr2=0x55ec8dc21fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc21fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 -- 172.21.15.3:0/1176317527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 msgr2=0x55ec8dc2f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 -- 172.21.15.3:0/1176317527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8db0eef0 con 0x55ec8dc224e0 2024-08-01T04:16:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f114be00640 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc21fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:03.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1149600640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f11340a2c00 0x7f11340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1149600640 1 -- 172.21.15.3:0/604843903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f11440efe60 con 0x7f114c10df30 2024-08-01T04:16:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.862+0000 7f1150a00640 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc22920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1140011540 tx=0x7f1140011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a37be27fa6cc24a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:03.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.863+0000 7f114be00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f11340a2c00 0x7f11340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.863+0000 7f1139c00640 1 -- 172.21.15.3:0/1176317527 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114000f640 con 0x55ec8dc224e0 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.863+0000 7f1139c00640 1 -- 172.21.15.3:0/1176317527 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114000fcc0 con 0x55ec8dc224e0 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.863+0000 7f1139c00640 1 -- 172.21.15.3:0/1176317527 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140019d60 con 0x55ec8dc224e0 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 -- 172.21.15.3:0/1176317527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 msgr2=0x55ec8dc22920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc22920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1140011540 tx=0x7f1140011a00 comp rx=0 tx=0).stop 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f114be00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f11340a2c00 0x7f11340a50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55ec8dc22f80 tx=0x7f113c004730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 -- 172.21.15.3:0/1176317527 shutdown_connections 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc22920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 --2- 172.21.15.3:0/1176317527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc21fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 -- 172.21.15.3:0/1176317527 >> 172.21.15.3:0/1176317527 conn(0x55ec8dbeaa00 msgr2=0x55ec8dc1eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 -- 172.21.15.3:0/1176317527 shutdown_connections 2024-08-01T04:16:03.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 -- 172.21.15.3:0/1176317527 wait complete. 2024-08-01T04:16:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.864+0000 7f1155e7a740 1 -- start start 2024-08-01T04:16:03.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 Processor -- start 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc2a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc2ff80 con 0x55ec8dc23a50 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ec8db43a80 con 0x55ec8dc2d280 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc30100 con 0x55ec8dc224e0 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f1155e7a740 10 client.0 _ll_register_callbacks cb 0x55ec8db0c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.865+0000 7f114be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51786/0 (socket says 172.21.15.3:51786) 2024-08-01T04:16:03.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 -- 172.21.15.3:0/590077212 learned_addr learned my addr 172.21.15.3:0/590077212 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f1150a00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc2a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f1151400640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 -- 172.21.15.3:0/590077212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 msgr2=0x55ec8dc2a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc2a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 -- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 msgr2=0x55ec8dc2ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 -- 172.21.15.3:0/590077212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8db27700 con 0x55ec8dc23a50 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f1151400640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f1150a00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc2a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f114be00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2a710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f113c021c40 tx=0x7f113c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bbc4a989ea4ddc server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.866+0000 7f112e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.867+0000 7f112e000640 1 -- 172.21.15.3:0/590077212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c02e060 con 0x55ec8dc23a50 2024-08-01T04:16:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.867+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114000fcc0 con 0x55ec8dc23a50 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.867+0000 7f112e000640 1 -- 172.21.15.3:0/590077212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f113c00f7d0 con 0x55ec8dc23a50 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.867+0000 7f112e000640 1 -- 172.21.15.3:0/590077212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f113c00f9d0 con 0x55ec8dc23a50 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.867+0000 7f1155e7a740 10 client.14874 fetch_fsmap learned FSMap version 25 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.867+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec8d8fa2b0 con 0x55ec8dc23a50 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f112e000640 1 -- 172.21.15.3:0/590077212 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114000fcc0 con 0x55ec8dc23a50 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f112e000640 1 -- 172.21.15.3:0/590077212 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f113c032030 con 0x55ec8dc23a50 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1155e7a740 10 client.14874 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1155e7a740 10 client.14874 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1155e7a740 20 client.14874 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1155e7a740 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dc3e190 0x55ec8dc3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1155e7a740 4 client.14874 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec8db95eb0 con 0x55ec8dc3e190 2024-08-01T04:16:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.868+0000 7f1150a00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dc3e190 0x55ec8dc3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.869+0000 7f1150a00640 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dc3e190 0x55ec8dc3f600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f268f5e5c83c23 server_cookie=16bd93b095605b7f in_seq=0 out_seq=0 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.869+0000 7f112e000640 10 client.14874 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f112e000640 1 -- 172.21.15.3:0/590077212 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec8db95eb0 con 0x55ec8dc3e190 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f112e000640 10 client.14874 handle_command_reply: tid=0 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 client.14874 shutdown 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 msgr2=0x55ec8dc2a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2a710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f113c021c40 tx=0x7f113c005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 shutdown_connections 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2a710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dc3e190 0x55ec8dc3f600 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 --2- 172.21.15.3:0/590077212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc2a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 >> 172.21.15.3:0/590077212 conn(0x55ec8dbeaa00 msgr2=0x55ec8dc24870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.870+0000 7f112e000640 0 client.14874 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.871+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 shutdown_connections 2024-08-01T04:16:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.871+0000 7f1155e7a740 1 -- 172.21.15.3:0/590077212 wait complete. 2024-08-01T04:16:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.871+0000 7f1155e7a740 20 client.14874 trim_cache size 0 max 16384 2024-08-01T04:16:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.890+0000 7f1155e7a740 1 Processor -- start 2024-08-01T04:16:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.890+0000 7f1155e7a740 1 -- start start 2024-08-01T04:16:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc21cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1155e7a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc3d940 con 0x55ec8dc2d280 2024-08-01T04:16:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1155e7a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc3e010 con 0x55ec8dc23a50 2024-08-01T04:16:03.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1155e7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc3e6e0 con 0x55ec8dc224e0 2024-08-01T04:16:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f114be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1151400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f114be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:55224/0 (socket says 172.21.15.3:55224) 2024-08-01T04:16:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f114be00640 1 -- 172.21.15.3:0/2823366367 learned_addr learned my addr 172.21.15.3:0/2823366367 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.891+0000 7f1150a00640 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc21cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f114be00640 1 -- 172.21.15.3:0/2823366367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 msgr2=0x55ec8dc21cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f114be00640 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc21cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f114be00640 1 -- 172.21.15.3:0/2823366367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 msgr2=0x55ec8dc2ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f114be00640 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f114be00640 1 -- 172.21.15.3:0/2823366367 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8dc3bbb0 con 0x55ec8dc23a50 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f114be00640 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2e670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f113c021c10 tx=0x7f113c010ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7dae7256996480 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f1139200640 1 -- 172.21.15.3:0/2823366367 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c00f7d0 con 0x55ec8dc23a50 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f1139200640 1 -- 172.21.15.3:0/2823366367 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c01fc30 con 0x55ec8dc23a50 2024-08-01T04:16:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.892+0000 7f1139200640 1 -- 172.21.15.3:0/2823366367 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0298c0 con 0x55ec8dc23a50 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 -- 172.21.15.3:0/2823366367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 msgr2=0x55ec8dc2e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2e670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f113c021c10 tx=0x7f113c010ce0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 -- 172.21.15.3:0/2823366367 shutdown_connections 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc2d280 0x55ec8dc2ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc23a50 0x55ec8dc2e670 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2823366367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc224e0 0x55ec8dc21cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 -- 172.21.15.3:0/2823366367 >> 172.21.15.3:0/2823366367 conn(0x55ec8dbeaa00 msgr2=0x55ec8dc25560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 -- 172.21.15.3:0/2823366367 shutdown_connections 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.893+0000 7f1155e7a740 1 -- 172.21.15.3:0/2823366367 wait complete. 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 -- start start 2024-08-01T04:16:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 Processor -- start 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 0x55ec8dc47850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc23a50 0x55ec8dc47d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc27450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc3f480 con 0x55ec8dc224e0 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc3edb0 con 0x55ec8dc2d280 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.894+0000 7f1155e7a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec8dc3fb50 con 0x55ec8dc23a50 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1155e7a740 10 client.0 _ll_register_callbacks cb 0x7f113c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 0x55ec8dc47850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 0x55ec8dc47850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51802/0 (socket says 172.21.15.3:51802) 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 -- 172.21.15.3:0/2041657768 learned_addr learned my addr 172.21.15.3:0/2041657768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f114be00640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc23a50 0x55ec8dc47d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1151400640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc27450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 -- 172.21.15.3:0/2041657768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc23a50 msgr2=0x55ec8dc47d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc23a50 0x55ec8dc47d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 -- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 msgr2=0x55ec8dc27450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc27450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1150a00640 1 -- 172.21.15.3:0/2041657768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8dc279c0 con 0x55ec8dc224e0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.895+0000 7f1151400640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f1150a00640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 0x55ec8dc47850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1140009d30 tx=0x7f1140019980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85b36af575ea60 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f112e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f112e000640 1 -- 172.21.15.3:0/2041657768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140019e30 con 0x55ec8dc224e0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f112e000640 1 -- 172.21.15.3:0/2041657768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f11400052c0 con 0x55ec8dc224e0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f113c01fc30 con 0x55ec8dc224e0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f112e000640 1 -- 172.21.15.3:0/2041657768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11400054e0 con 0x55ec8dc224e0 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f1155e7a740 10 client.14880 fetch_fsmap learned FSMap version 25 2024-08-01T04:16:03.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.896+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec8dc1de20 con 0x55ec8dc224e0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f112e000640 1 -- 172.21.15.3:0/2041657768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c01fc30 con 0x55ec8dc224e0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f112e000640 1 -- 172.21.15.3:0/2041657768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f11400035f0 con 0x55ec8dc224e0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f1155e7a740 10 client.14880 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f1155e7a740 10 client.14880 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f1155e7a740 20 client.14880 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dcacad0 0x55ec8dcab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f1155e7a740 4 client.14880 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "status"}) -- 0x55ec8db903c0 con 0x55ec8dcacad0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.897+0000 7f114be00640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dcacad0 0x55ec8dcab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f114be00640 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dcacad0 0x55ec8dcab7e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41e54a4b614d24de server_cookie=aaa2ed8732a77a3e in_seq=0 out_seq=0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f112e000640 10 client.14880 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f112e000640 1 -- 172.21.15.3:0/2041657768 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ec8db903c0 con 0x55ec8dcacad0 2024-08-01T04:16:03.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f112e000640 10 client.14880 handle_command_reply: tid=0 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 client.14880 shutdown 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 msgr2=0x55ec8dc47850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 0x55ec8dc47850 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1140009d30 tx=0x7f1140019980 comp rx=0 tx=0).stop 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 shutdown_connections 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec8dc2d280 0x55ec8dc27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55ec8dcacad0 0x55ec8dcab7e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec8dc23a50 0x55ec8dc47d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 --2- 172.21.15.3:0/2041657768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec8dc224e0 0x55ec8dc47850 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 >> 172.21.15.3:0/2041657768 conn(0x55ec8dbeaa00 msgr2=0x55ec8dc286e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.898+0000 7f112e000640 0 client.14880 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.899+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 shutdown_connections 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.899+0000 7f1155e7a740 1 -- 172.21.15.3:0/2041657768 wait complete. 2024-08-01T04:16:03.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.899+0000 7f1155e7a740 20 client.14880 trim_cache size 0 max 16384 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24506, 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 116.304858393, 2024-08-01T04:16:03.907 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 25, 2024-08-01T04:16:03.908 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-08-01T04:16:03.908 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 86, 2024-08-01T04:16:03.908 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 155.77854995300001 2024-08-01T04:16:03.908 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:03.909 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:03 smithi160 ceph-mon[37224]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 518 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f11340a2c00 msgr2=0x7f11340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f11340a2c00 0x7f11340a50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55ec8dc22f80 tx=0x7f113c004730 comp rx=0 tx=0).stop 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 msgr2=0x7f114c19ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c19ead0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1144005e30 tx=0x7f114402f630 comp rx=0 tx=0).stop 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 shutdown_connections 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f11340a2c00 0x7f11340a50c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f114c10df30 0x7f114c19ead0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f114c074c70 0x7f114c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 --2- 172.21.15.3:0/604843903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f114c074350 0x7f114c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:03.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 >> 172.21.15.3:0/604843903 conn(0x7f114c0fd410 msgr2=0x7f114c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:03.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 shutdown_connections 2024-08-01T04:16:03.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:03.900+0000 7f1151e00640 1 -- 172.21.15.3:0/604843903 wait complete. 2024-08-01T04:16:04.766 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.a status 2024-08-01T04:16:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:05.651 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:05 smithi003 ceph-mon[32262]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:05.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:05 smithi163 ceph-mon[37383]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:05.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:05 smithi160 ceph-mon[37224]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 -- 172.21.15.3:0/2222159483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80076450 msgr2=0x7fbd80076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 --2- 172.21.15.3:0/2222159483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80076450 0x7fbd80076830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009860 tx=0x7fbd7002a340 comp rx=0 tx=0).stop 2024-08-01T04:16:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 -- 172.21.15.3:0/2222159483 shutdown_connections 2024-08-01T04:16:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 --2- 172.21.15.3:0/2222159483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80112760 0x7fbd80114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 --2- 172.21.15.3:0/2222159483 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd80112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 --2- 172.21.15.3:0/2222159483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80076450 0x7fbd80076830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 -- 172.21.15.3:0/2222159483 >> 172.21.15.3:0/2222159483 conn(0x7fbd800ff7c0 msgr2=0x7fbd80101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 -- 172.21.15.3:0/2222159483 shutdown_connections 2024-08-01T04:16:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.902+0000 7fbd87200640 1 -- 172.21.15.3:0/2222159483 wait complete. 2024-08-01T04:16:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 Processor -- start 2024-08-01T04:16:05.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 -- start start 2024-08-01T04:16:05.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80076450 0x7fbd801a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd801a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80112760 0x7fbd8019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801182f0 con 0x7fbd80076450 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80118170 con 0x7fbd80076d70 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.904+0000 7fbd87200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80118470 con 0x7fbd80112760 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80076450 0x7fbd801a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd801a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd801a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:55250/0 (socket says 172.21.15.3:55250) 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7f400640 1 -- 172.21.15.3:0/3981751427 learned_addr learned my addr 172.21.15.3:0/3981751427 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:05.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd85200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80112760 0x7fbd8019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7fe00640 1 -- 172.21.15.3:0/3981751427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80112760 msgr2=0x7fbd8019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80112760 0x7fbd8019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7fe00640 1 -- 172.21.15.3:0/3981751427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 msgr2=0x7fbd801a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd801a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7fe00640 1 -- 172.21.15.3:0/3981751427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8019ed10 con 0x7fbd80076450 2024-08-01T04:16:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.905+0000 7fbd7f400640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd801a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.906+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80076450 0x7fbd801a4420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002a860 tx=0x7fbd70033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.906+0000 7fbd7cc00640 1 -- 172.21.15.3:0/3981751427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70037040 con 0x7fbd80076450 2024-08-01T04:16:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.906+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8019efa0 con 0x7fbd80076450 2024-08-01T04:16:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.906+0000 7fbd7cc00640 1 -- 172.21.15.3:0/3981751427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd70033d50 con 0x7fbd80076450 2024-08-01T04:16:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.906+0000 7fbd7cc00640 1 -- 172.21.15.3:0/3981751427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70003810 con 0x7fbd80076450 2024-08-01T04:16:05.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.906+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801ab2f0 con 0x7fbd80076450 2024-08-01T04:16:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.909+0000 7fbd89df1740 1 Processor -- start 2024-08-01T04:16:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 -- start start 2024-08-01T04:16:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618667f8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618668030b0 0x5618668054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56186675e940 con 0x5618667f9880 2024-08-01T04:16:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5618666f69a0 con 0x5618667f8310 2024-08-01T04:16:05.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd89df1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56186675a900 con 0x5618668030b0 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd7cc00640 1 -- 172.21.15.3:0/3981751427 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fbd70042020 con 0x7fbd80076450 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd85200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd85200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51834/0 (socket says 172.21.15.3:51834) 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd85200640 1 -- 172.21.15.3:0/402074153 learned_addr learned my addr 172.21.15.3:0/402074153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.910+0000 7fbd7f400640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618668030b0 0x5618668054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618667f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7f400640 1 -- 172.21.15.3:0/402074153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 msgr2=0x5618667f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7f400640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618667f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7f400640 1 -- 172.21.15.3:0/402074153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 msgr2=0x5618667f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7f400640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7f400640 1 -- 172.21.15.3:0/402074153 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618666e4ef0 con 0x5618668030b0 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618667f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd85200640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:05.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.911+0000 7fbd7f400640 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618668030b0 0x5618668054a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd8019f720 tx=0x7fbd68011cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b84b7b7466011768 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd75600640 1 -- 172.21.15.3:0/402074153 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68011250 con 0x5618668030b0 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd75600640 1 -- 172.21.15.3:0/402074153 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6800f970 con 0x5618668030b0 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd75600640 1 -- 172.21.15.3:0/402074153 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68005700 con 0x5618668030b0 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd7cc00640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd600a2b80 0x7fbd600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd7cc00640 1 -- 172.21.15.3:0/3981751427 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7fbd700ee670 con 0x7fbd80076450 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd85200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd600a2b80 0x7fbd600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd89df1740 1 -- 172.21.15.3:0/402074153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618668030b0 msgr2=0x5618668054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.912+0000 7fbd89df1740 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618668030b0 0x5618668054a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd8019f720 tx=0x7fbd68011cf0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 -- 172.21.15.3:0/402074153 shutdown_connections 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618668030b0 0x5618668054a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618667f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 --2- 172.21.15.3:0/402074153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 -- 172.21.15.3:0/402074153 >> 172.21.15.3:0/402074153 conn(0x5618667c0a00 msgr2=0x5618667f49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 -- 172.21.15.3:0/402074153 shutdown_connections 2024-08-01T04:16:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 -- 172.21.15.3:0/402074153 wait complete. 2024-08-01T04:16:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd85200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd600a2b80 0x7fbd600a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009e90 tx=0x7fbd78009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.913+0000 7fbd89df1740 1 -- start start 2024-08-01T04:16:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 Processor -- start 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f8310 0x561866802b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x561866800690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561866805db0 con 0x5618667f9880 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561866719a80 con 0x5618668030b0 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561866805f30 con 0x5618667f8310 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd89df1740 10 client.0 _ll_register_callbacks cb 0x5618666e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:05.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.914+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51838/0 (socket says 172.21.15.3:51838) 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 -- 172.21.15.3:0/216876314 learned_addr learned my addr 172.21.15.3:0/216876314 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f8310 0x561866802b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd85200640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x561866800690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 -- 172.21.15.3:0/216876314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f8310 msgr2=0x561866802b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f8310 0x561866802b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 -- 172.21.15.3:0/216876314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 msgr2=0x561866800690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x561866800690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 -- 172.21.15.3:0/216876314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618666fd700 con 0x5618667f9880 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd7f400640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667fff00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005360 tx=0x7fbd680112f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4a74ad513c4b6a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.915+0000 7fbd6e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd6e000640 1 -- 172.21.15.3:0/216876314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68028020 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd6e000640 1 -- 172.21.15.3:0/216876314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7fbd68005810 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6800f970 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd6e000640 1 -- 172.21.15.3:0/216876314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd68033050 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd89df1740 10 client.14892 fetch_fsmap learned FSMap version 25 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618664d02b0 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.916+0000 7fbd6e000640 1 -- 172.21.15.3:0/216876314 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6800f970 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd6e000640 1 -- 172.21.15.3:0/216876314 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd68018070 con 0x5618667f9880 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd89df1740 10 client.14892 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd89df1740 10 client.14892 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd89df1740 20 client.14892 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd89df1740 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866813b00 0x561866814f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd89df1740 4 client.14892 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56186676beb0 con 0x561866813b00 2024-08-01T04:16:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.917+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866813b00 0x561866814f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.918+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866813b00 0x561866814f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee4dd6003705a293 server_cookie=2e767b728f6bf98e in_seq=0 out_seq=0 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.918+0000 7fbd6e000640 10 client.14892 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd6e000640 1 -- 172.21.15.3:0/216876314 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56186676beb0 con 0x561866813b00 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd6e000640 10 client.14892 handle_command_reply: tid=0 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 client.14892 shutdown 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 msgr2=0x5618667fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667fff00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005360 tx=0x7fbd680112f0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 shutdown_connections 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866813b00 0x561866814f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x561866800690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f9880 0x5618667fff00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 --2- 172.21.15.3:0/216876314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f8310 0x561866802b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 >> 172.21.15.3:0/216876314 conn(0x5618667c0a00 msgr2=0x5618667fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd6e000640 0 client.14892 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 shutdown_connections 2024-08-01T04:16:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.919+0000 7fbd89df1740 1 -- 172.21.15.3:0/216876314 wait complete. 2024-08-01T04:16:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.920+0000 7fbd89df1740 20 client.14892 trim_cache size 0 max 16384 2024-08-01T04:16:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.939+0000 7fbd89df1740 1 Processor -- start 2024-08-01T04:16:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.939+0000 7fbd89df1740 1 -- start start 2024-08-01T04:16:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618668141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x5618667feee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 0x5618667fd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd89df1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5618668132b0 con 0x5618668030b0 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd89df1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561866813980 con 0x5618667f8310 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd89df1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561866814050 con 0x5618667f9880 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 0x5618667fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 0x5618667fd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51848/0 (socket says 172.21.15.3:51848) 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 -- 172.21.15.3:0/1155274669 learned_addr learned my addr 172.21.15.3:0/1155274669 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd7f400640 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x5618667feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618668141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 -- 172.21.15.3:0/1155274669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 msgr2=0x5618667feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x5618667feee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 -- 172.21.15.3:0/1155274669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 msgr2=0x5618668141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618668141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd85200640 1 -- 172.21.15.3:0/1155274669 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618667fe450 con 0x5618668030b0 2024-08-01T04:16:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.940+0000 7fbd7f400640 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x5618667feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.941+0000 7fbd85200640 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 0x5618667fd180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbd780043a0 tx=0x7fbd7801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853eb9db1d3f1e86 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.941+0000 7fbd74c00640 1 -- 172.21.15.3:0/1155274669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78004490 con 0x5618668030b0 2024-08-01T04:16:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.941+0000 7fbd74c00640 1 -- 172.21.15.3:0/1155274669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7802a960 con 0x5618668030b0 2024-08-01T04:16:05.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.941+0000 7fbd74c00640 1 -- 172.21.15.3:0/1155274669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7801faf0 con 0x5618668030b0 2024-08-01T04:16:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 -- 172.21.15.3:0/1155274669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 msgr2=0x5618667fd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 0x5618667fd180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbd780043a0 tx=0x7fbd7801a000 comp rx=0 tx=0).stop 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 -- 172.21.15.3:0/1155274669 shutdown_connections 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618668030b0 0x5618667fd180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x5618667feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 --2- 172.21.15.3:0/1155274669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618667f8310 0x5618668141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 -- 172.21.15.3:0/1155274669 >> 172.21.15.3:0/1155274669 conn(0x5618667c0a00 msgr2=0x5618667fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 -- 172.21.15.3:0/1155274669 shutdown_connections 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.942+0000 7fbd89df1740 1 -- 172.21.15.3:0/1155274669 wait complete. 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 -- start start 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 Processor -- start 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 0x56186681d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x56186681d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x56186681db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561866814df0 con 0x5618667f8310 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561866814720 con 0x5618668030b0 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.943+0000 7fbd89df1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5618668154c0 con 0x5618667f9880 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd89df1740 10 client.0 _ll_register_callbacks cb 0x7fbd6800fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 0x56186681d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 0x56186681d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51864/0 (socket says 172.21.15.3:51864) 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 -- 172.21.15.3:0/2537996724 learned_addr learned my addr 172.21.15.3:0/2537996724 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7f400640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x56186681d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd85200640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x56186681db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 -- 172.21.15.3:0/2537996724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 msgr2=0x56186681d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x56186681d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 -- 172.21.15.3:0/2537996724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 msgr2=0x56186681db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:05.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x56186681db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7fe00640 1 -- 172.21.15.3:0/2537996724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561866818240 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.944+0000 7fbd7f400640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x56186681d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd7fe00640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 0x56186681d0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd700baac0 tx=0x7fbd700308f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac3917aaffc16ca server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd6e000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd6e000640 1 -- 172.21.15.3:0/2537996724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700c3a80 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd6e000640 1 -- 172.21.15.3:0/2537996724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7fbd700b7480 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7802a960 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd6e000640 1 -- 172.21.15.3:0/2537996724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd70030550 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd89df1740 10 client.14898 fetch_fsmap learned FSMap version 25 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56186687e6b0 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.945+0000 7fbd6e000640 1 -- 172.21.15.3:0/2537996724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7802a960 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd6e000640 1 -- 172.21.15.3:0/2537996724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbd7003a070 con 0x5618667f8310 2024-08-01T04:16:05.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd89df1740 10 client.14898 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:16:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd89df1740 10 client.14898 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-01T04:16:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd89df1740 20 client.14898 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd89df1740 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866883300 0x561866882010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd89df1740 4 client.14898 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:16:05.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "status"}) -- 0x5618667663c0 con 0x561866883300 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.946+0000 7fbd7f400640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866883300 0x561866882010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd7f400640 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866883300 0x561866882010 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e14ea1205d391c5 server_cookie=300c479fac340730 in_seq=0 out_seq=0 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd6e000640 10 client.14898 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd6e000640 1 -- 172.21.15.3:0/2537996724 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5618667663c0 con 0x561866883300 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd6e000640 10 client.14898 handle_command_reply: tid=0 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd89df1740 1 client.14898 shutdown 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 msgr2=0x56186681d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.947+0000 7fbd89df1740 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 0x56186681d0e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd700baac0 tx=0x7fbd700308f0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 shutdown_connections 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561866883300 0x561866882010 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5618668030b0 0x56186681db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618667f9880 0x56186681d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 --2- 172.21.15.3:0/2537996724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5618667f8310 0x56186681d0e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 >> 172.21.15.3:0/2537996724 conn(0x5618667c0a00 msgr2=0x5618667feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd6e000640 0 client.14898 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 shutdown_connections 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 1 -- 172.21.15.3:0/2537996724 wait complete. 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.948+0000 7fbd89df1740 20 client.14898 trim_cache size 0 max 16384 2024-08-01T04:16:05.953 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24520, 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 119.36388358000001, 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 25, 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 86, 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 147.26898334200001 2024-08-01T04:16:05.954 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd600a2b80 msgr2=0x7fbd600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd600a2b80 0x7fbd600a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009e90 tx=0x7fbd78009260 comp rx=0 tx=0).stop 2024-08-01T04:16:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80076450 msgr2=0x7fbd801a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80076450 0x7fbd801a4420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd7002a860 tx=0x7fbd70033b60 comp rx=0 tx=0).stop 2024-08-01T04:16:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 shutdown_connections 2024-08-01T04:16:05.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fbd600a2b80 0x7fbd600a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd80112760 0x7fbd8019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd80076d70 0x7fbd801a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 --2- 172.21.15.3:0/3981751427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd80076450 0x7fbd801a4420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 >> 172.21.15.3:0/3981751427 conn(0x7fbd800ff7c0 msgr2=0x7fbd80101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 shutdown_connections 2024-08-01T04:16:05.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:05.949+0000 7fbd87200640 1 -- 172.21.15.3:0/3981751427 wait complete. 2024-08-01T04:16:06.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-08-01T04:16:07.398 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:07.794 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:07 smithi003 ceph-mon[32262]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:07.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:07 smithi160 ceph-mon[37224]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:07 smithi163 ceph-mon[37383]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-08-01T04:16:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.067+0000 7f9312000640 1 -- 172.21.15.3:0/1380846043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 msgr2=0x7f930c114990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.067+0000 7f9312000640 1 --2- 172.21.15.3:0/1380846043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 0x7f930c114990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9304009be0 tx=0x7f930402a6d0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.067+0000 7f9312000640 1 -- 172.21.15.3:0/1380846043 shutdown_connections 2024-08-01T04:16:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.067+0000 7f9312000640 1 --2- 172.21.15.3:0/1380846043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 0x7f930c114990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.068+0000 7f9312000640 1 --2- 172.21.15.3:0/1380846043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.068+0000 7f9312000640 1 --2- 172.21.15.3:0/1380846043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 0x7f930c076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.068+0000 7f9312000640 1 -- 172.21.15.3:0/1380846043 >> 172.21.15.3:0/1380846043 conn(0x7f930c0ff600 msgr2=0x7f930c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.068+0000 7f9312000640 1 -- 172.21.15.3:0/1380846043 shutdown_connections 2024-08-01T04:16:08.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.068+0000 7f9312000640 1 -- 172.21.15.3:0/1380846043 wait complete. 2024-08-01T04:16:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.069+0000 7f9312000640 1 Processor -- start 2024-08-01T04:16:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.069+0000 7f9312000640 1 -- start start 2024-08-01T04:16:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9312000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 0x7f930c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9312000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9312000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 0x7f930c1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9312000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1180e0 con 0x7f930c076450 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9312000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f930c117f60 con 0x7f930c1125a0 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9312000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f930c118260 con 0x7f930c076d70 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f930be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9310c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 0x7f930c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f930be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c19eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:33134/0 (socket says 172.21.15.3:33134) 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f930be00640 1 -- 172.21.15.3:0/629133375 learned_addr learned my addr 172.21.15.3:0/629133375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.070+0000 7f9311600640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 0x7f930c1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f930be00640 1 -- 172.21.15.3:0/629133375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 msgr2=0x7f930c1a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f930be00640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 0x7f930c1a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f930be00640 1 -- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 msgr2=0x7f930c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f930be00640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 0x7f930c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f930be00640 1 -- 172.21.15.3:0/629133375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1a3590 con 0x7f930c076d70 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f9310c00640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 0x7f930c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f930be00640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c19eb20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9f0 tx=0x7f92f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f9309600640 1 -- 172.21.15.3:0/629133375 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f800ccc0 con 0x7f930c076d70 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f9309600640 1 -- 172.21.15.3:0/629133375 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f800ce60 con 0x7f930c076d70 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c1a3880 con 0x7f930c076d70 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.071+0000 7f9309600640 1 -- 172.21.15.3:0/629133375 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8010690 con 0x7f930c076d70 2024-08-01T04:16:08.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.072+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1ab160 con 0x7f930c076d70 2024-08-01T04:16:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.076+0000 7f93161bf740 1 Processor -- start 2024-08-01T04:16:08.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.076+0000 7f93161bf740 1 -- start start 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f9309600640 1 -- 172.21.15.3:0/629133375 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f92f80046c0 con 0x7f930c076d70 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f93161bf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded855fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f93161bf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded856920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f93161bf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded863670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f93161bf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fded7bc940 con 0x55fded861280 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f93161bf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fded7549a0 con 0x55fded857a50 2024-08-01T04:16:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f93161bf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fded7b8900 con 0x55fded8564e0 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f9311600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded855fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f930be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded863670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f9311600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded855fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:55318/0 (socket says 172.21.15.3:55318) 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f9311600640 1 -- 172.21.15.3:0/1786396753 learned_addr learned my addr 172.21.15.3:0/1786396753 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.077+0000 7f9310c00640 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded856920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.078+0000 7f9311600640 1 -- 172.21.15.3:0/1786396753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 msgr2=0x55fded856920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.078+0000 7f9311600640 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded856920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.078+0000 7f9311600640 1 -- 172.21.15.3:0/1786396753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 msgr2=0x55fded863670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.078+0000 7f9311600640 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded863670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.078+0000 7f9311600640 1 -- 172.21.15.3:0/1786396753 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fded742ef0 con 0x55fded857a50 2024-08-01T04:16:08.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.078+0000 7f9311600640 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded855fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9304004af0 tx=0x7f9304030a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5bccc0aa527235 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f92fdc00640 1 -- 172.21.15.3:0/1786396753 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304041030 con 0x55fded857a50 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f92fdc00640 1 -- 172.21.15.3:0/1786396753 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304030c60 con 0x55fded857a50 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f92fdc00640 1 -- 172.21.15.3:0/1786396753 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304036040 con 0x55fded857a50 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f9309600640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f92f40a2c00 0x7f92f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f9309600640 1 -- 172.21.15.3:0/629133375 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f92f80cf3f0 con 0x7f930c076d70 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f9310c00640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f92f40a2c00 0x7f92f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f93161bf740 1 -- 172.21.15.3:0/1786396753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 msgr2=0x55fded855fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.079+0000 7f93161bf740 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded855fa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9304004af0 tx=0x7f9304030a30 comp rx=0 tx=0).stop 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 -- 172.21.15.3:0/1786396753 shutdown_connections 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded863670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded856920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 --2- 172.21.15.3:0/1786396753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded855fa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 -- 172.21.15.3:0/1786396753 >> 172.21.15.3:0/1786396753 conn(0x55fded81ea00 msgr2=0x55fded852b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 -- 172.21.15.3:0/1786396753 shutdown_connections 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 -- 172.21.15.3:0/1786396753 wait complete. 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f9310c00640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f92f40a2c00 0x7f92f40a50c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55fded8539d0 tx=0x7f9300009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:08.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.080+0000 7f93161bf740 1 -- start start 2024-08-01T04:16:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 Processor -- start 2024-08-01T04:16:08.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded85e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded85e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fded863f20 con 0x55fded861280 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fded777a80 con 0x55fded857a50 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fded8640a0 con 0x55fded8564e0 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.081+0000 7f93161bf740 10 client.0 _ll_register_callbacks cb 0x55fded740140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9311600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded85e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f930be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded85e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9311600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51906/0 (socket says 172.21.15.3:51906) 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9311600640 1 -- 172.21.15.3:0/1685015223 learned_addr learned my addr 172.21.15.3:0/1685015223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 -- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 msgr2=0x55fded85e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded85e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 -- 172.21.15.3:0/1685015223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 msgr2=0x55fded85ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 -- 172.21.15.3:0/1685015223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fded75b700 con 0x55fded8564e0 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f9310c00640 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded85e1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93000035f0 tx=0x7f9300004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16202db0dcb2760 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:08.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.082+0000 7f92f2000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f92f2000640 1 -- 172.21.15.3:0/1685015223 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930002c060 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f92f2000640 1 -- 172.21.15.3:0/1685015223 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f930000f970 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9304030c60 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f92f2000640 1 -- 172.21.15.3:0/1685015223 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f930000fb70 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f93161bf740 10 client.25021 fetch_fsmap learned FSMap version 25 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fded52e2b0 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.083+0000 7f92f2000640 1 -- 172.21.15.3:0/1685015223 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304030c60 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f92f2000640 1 -- 172.21.15.3:0/1685015223 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9300030030 con 0x55fded8564e0 2024-08-01T04:16:08.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f93161bf740 10 client.25021 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:16:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f93161bf740 10 client.25021 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-01T04:16:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f93161bf740 20 client.25021 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f93161bf740 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded872070 0x55fded8734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f93161bf740 4 client.25021 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:08.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fded7c9eb0 con 0x55fded872070 2024-08-01T04:16:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.084+0000 7f930be00640 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded872070 0x55fded8734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.085+0000 7f930be00640 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded872070 0x55fded8734e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b80ea68ab394448 server_cookie=2bf812d2a757d8bf in_seq=0 out_seq=0 2024-08-01T04:16:08.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.085+0000 7f92f2000640 10 client.25021 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.085+0000 7f92f2000640 1 -- 172.21.15.3:0/1685015223 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fded7c9eb0 con 0x55fded872070 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.085+0000 7f92f2000640 10 client.25021 handle_command_reply: tid=0 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 client.25021 shutdown 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 msgr2=0x55fded85e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded85e1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93000035f0 tx=0x7f9300004730 comp rx=0 tx=0).stop 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 shutdown_connections 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded872070 0x55fded8734e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded85e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 --2- 172.21.15.3:0/1685015223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded85e1b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f92f2000640 0 client.25021 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:08.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 >> 172.21.15.3:0/1685015223 conn(0x55fded81ea00 msgr2=0x55fded858870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 shutdown_connections 2024-08-01T04:16:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 1 -- 172.21.15.3:0/1685015223 wait complete. 2024-08-01T04:16:08.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.086+0000 7f93161bf740 20 client.25021 trim_cache size 0 max 16384 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.106+0000 7f93161bf740 1 Processor -- start 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.106+0000 7f93161bf740 1 -- start start 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f93161bf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded8719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f93161bf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded852d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f93161bf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f93161bf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fded871820 con 0x55fded861280 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f93161bf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fded871ef0 con 0x55fded857a50 2024-08-01T04:16:08.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f93161bf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fded8725c0 con 0x55fded8564e0 2024-08-01T04:16:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f9311600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f9311600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51908/0 (socket says 172.21.15.3:51908) 2024-08-01T04:16:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f9311600640 1 -- 172.21.15.3:0/2601965921 learned_addr learned my addr 172.21.15.3:0/2601965921 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f9310c00640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded8719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.107+0000 7f930be00640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded852d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9310c00640 1 -- 172.21.15.3:0/2601965921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 msgr2=0x55fded852d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9310c00640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded852d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9310c00640 1 -- 172.21.15.3:0/2601965921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 msgr2=0x55fded85e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9310c00640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9310c00640 1 -- 172.21.15.3:0/2601965921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fded859030 con 0x55fded8564e0 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f930be00640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded852d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9311600640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f9310c00640 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded8719a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f9300005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e73aa0c7d75fb30 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f92fd200640 1 -- 172.21.15.3:0/2601965921 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930000f970 con 0x55fded8564e0 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f92fd200640 1 -- 172.21.15.3:0/2601965921 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930001e6c0 con 0x55fded8564e0 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.108+0000 7f92fd200640 1 -- 172.21.15.3:0/2601965921 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300011440 con 0x55fded8564e0 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 -- 172.21.15.3:0/2601965921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 msgr2=0x55fded8719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded8719a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f9300005c50 comp rx=0 tx=0).stop 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 -- 172.21.15.3:0/2601965921 shutdown_connections 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded861280 0x55fded85e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded857a50 0x55fded852d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 --2- 172.21.15.3:0/2601965921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded8564e0 0x55fded8719a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 -- 172.21.15.3:0/2601965921 >> 172.21.15.3:0/2601965921 conn(0x55fded81ea00 msgr2=0x55fded852610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 -- 172.21.15.3:0/2601965921 shutdown_connections 2024-08-01T04:16:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.109+0000 7f93161bf740 1 -- 172.21.15.3:0/2601965921 wait complete. 2024-08-01T04:16:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 -- start start 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 Processor -- start 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded8564e0 0x55fded87be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 0x55fded879950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded861280 0x55fded879e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fded873360 con 0x55fded857a50 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fded872c90 con 0x55fded8564e0 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fded873a30 con 0x55fded861280 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.110+0000 7f93161bf740 10 client.0 _ll_register_callbacks cb 0x7f9300004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f930be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 0x55fded879950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9310c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded8564e0 0x55fded87be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f930be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 0x55fded879950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51920/0 (socket says 172.21.15.3:51920) 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f930be00640 1 -- 172.21.15.3:0/3267714522 learned_addr learned my addr 172.21.15.3:0/3267714522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9311600640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded861280 0x55fded879e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9310c00640 1 -- 172.21.15.3:0/3267714522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded861280 msgr2=0x55fded879e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9310c00640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded861280 0x55fded879e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9310c00640 1 -- 172.21.15.3:0/3267714522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 msgr2=0x55fded879950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9310c00640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 0x55fded879950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f9310c00640 1 -- 172.21.15.3:0/3267714522 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fded87a6c0 con 0x55fded8564e0 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.111+0000 7f930be00640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 0x55fded879950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f9310c00640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded8564e0 0x55fded87be80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9300005f70 tx=0x7f9300012f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284045f306603dd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:08.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f92f2000640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f92f2000640 1 -- 172.21.15.3:0/3267714522 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930003e030 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930001e6c0 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f92f2000640 1 -- 172.21.15.3:0/3267714522 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7f9300018040 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f92f2000640 1 -- 172.21.15.3:0/3267714522 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9300048050 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f93161bf740 10 client.25043 fetch_fsmap learned FSMap version 25 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.112+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fded8dc780 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f92f2000640 1 -- 172.21.15.3:0/3267714522 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930001e6c0 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f92f2000640 1 -- 172.21.15.3:0/3267714522 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9300039070 con 0x55fded8564e0 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f93161bf740 10 client.25043 fetch_fsmap finished waiting for FSMap version 25 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f93161bf740 10 client.25043 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f93161bf740 20 client.25043 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f93161bf740 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded8e0fa0 0x55fded8dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f93161bf740 4 client.25043 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:16:08.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.113+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "status"}) -- 0x55fded7c43c0 con 0x55fded8e0fa0 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.114+0000 7f930be00640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded8e0fa0 0x55fded8dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.114+0000 7f930be00640 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded8e0fa0 0x55fded8dfcb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d52af5589a1381 server_cookie=5857b56e6b335ea8 in_seq=0 out_seq=0 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.114+0000 7f92f2000640 10 client.25043 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.114+0000 7f92f2000640 1 -- 172.21.15.3:0/3267714522 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55fded7c43c0 con 0x55fded8e0fa0 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.114+0000 7f92f2000640 10 client.25043 handle_command_reply: tid=0 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.114+0000 7f93161bf740 1 client.25043 shutdown 2024-08-01T04:16:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded8564e0 msgr2=0x55fded87be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded8564e0 0x55fded87be80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9300005f70 tx=0x7f9300012f80 comp rx=0 tx=0).stop 2024-08-01T04:16:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 shutdown_connections 2024-08-01T04:16:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fded8e0fa0 0x55fded8dfcb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fded861280 0x55fded879e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fded857a50 0x55fded879950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 --2- 172.21.15.3:0/3267714522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fded8564e0 0x55fded87be80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f92f2000640 0 client.25043 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 >> 172.21.15.3:0/3267714522 conn(0x55fded81ea00 msgr2=0x55fded858840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 shutdown_connections 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 1 -- 172.21.15.3:0/3267714522 wait complete. 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.115+0000 7f93161bf740 20 client.25043 trim_cache size 0 max 16384 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:08.121 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24548, 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 131.59599365, 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 25, 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 90, 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 86, 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 146.63859669999999 2024-08-01T04:16:08.122 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f92f40a2c00 msgr2=0x7f92f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f92f40a2c00 0x7f92f40a50c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55fded8539d0 tx=0x7f9300009260 comp rx=0 tx=0).stop 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 msgr2=0x7f930c19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c19eb20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9f0 tx=0x7f92f800edf0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 shutdown_connections 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7f92f40a2c00 0x7f92f40a50c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f930c1125a0 0x7f930c1a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f930c076d70 0x7f930c19eb20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 --2- 172.21.15.3:0/629133375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f930c076450 0x7f930c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 >> 172.21.15.3:0/629133375 conn(0x7f930c0ff600 msgr2=0x7f930c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 shutdown_connections 2024-08-01T04:16:08.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:08.116+0000 7f9312000640 1 -- 172.21.15.3:0/629133375 wait complete. 2024-08-01T04:16:09.026 INFO:teuthology.run_tasks:Running task sequential... 2024-08-01T04:16:09.037 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-01T04:16:09.037 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-01T04:16:09.037 INFO:teuthology.task.print:Enabling mgr modules 2024-08-01T04:16:09.037 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-08-01T04:16:09.038 INFO:teuthology.task.exec:Executing custom commands... 2024-08-01T04:16:09.038 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:16:09.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.254+0000 7fda99c00640 1 -- 172.21.15.3:0/3440229077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 msgr2=0x7fda941018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.254+0000 7fda99c00640 1 --2- 172.21.15.3:0/3440229077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 0x7fda941018e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009ca0 tx=0x7fda8c02a8c0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 -- 172.21.15.3:0/3440229077 shutdown_connections 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 --2- 172.21.15.3:0/3440229077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 0x7fda941018e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 --2- 172.21.15.3:0/3440229077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda941006a0 0x7fda94100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 --2- 172.21.15.3:0/3440229077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda940ff940 0x7fda940ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 -- 172.21.15.3:0/3440229077 >> 172.21.15.3:0/3440229077 conn(0x7fda940fb480 msgr2=0x7fda940fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:09.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 -- 172.21.15.3:0/3440229077 shutdown_connections 2024-08-01T04:16:09.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.255+0000 7fda99c00640 1 -- 172.21.15.3:0/3440229077 wait complete. 2024-08-01T04:16:09.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 Processor -- start 2024-08-01T04:16:09.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 -- start start 2024-08-01T04:16:09.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda940ff940 0x7fda94199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:09.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 0x7fda9419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:09.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 0x7fda941a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda9410f570 con 0x7fda941006a0 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fda9410f3f0 con 0x7fda940ff940 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.256+0000 7fda99c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fda9410f6f0 con 0x7fda94101060 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 0x7fda9419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 0x7fda9419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51942/0 (socket says 172.21.15.3:51942) 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 -- 172.21.15.3:0/3377631577 learned_addr learned my addr 172.21.15.3:0/3377631577 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93e00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda940ff940 0x7fda94199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda99200640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 0x7fda941a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 -- 172.21.15.3:0/3377631577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 msgr2=0x7fda941a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 0x7fda941a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 -- 172.21.15.3:0/3377631577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda940ff940 msgr2=0x7fda94199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda940ff940 0x7fda94199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 -- 172.21.15.3:0/3377631577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda941a17b0 con 0x7fda941006a0 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93e00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda940ff940 0x7fda94199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.257+0000 7fda93400640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 0x7fda9419a0d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.258+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8000cd80 con 0x7fda941006a0 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.258+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda800035f0 con 0x7fda941006a0 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.258+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda941a1aa0 con 0x7fda941006a0 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.258+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80010690 con 0x7fda941006a0 2024-08-01T04:16:09.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.258+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda941a1fa0 con 0x7fda941006a0 2024-08-01T04:16:09.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.259+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9410b450 con 0x7fda941006a0 2024-08-01T04:16:09.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.262+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fda800037a0 con 0x7fda941006a0 2024-08-01T04:16:09.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.263+0000 7fda90c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fda780a2b80 0x7fda780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:09.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.263+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7fda800cf450 con 0x7fda941006a0 2024-08-01T04:16:09.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.263+0000 7fda93e00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fda780a2b80 0x7fda780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:09.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.264+0000 7fda93e00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fda780a2b80 0x7fda780a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fda84009fc0 tx=0x7fda84005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:09.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.268+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fda8009c2f0 con 0x7fda941006a0 2024-08-01T04:16:09.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.525+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fda94062b20 con 0x7fda941006a0 2024-08-01T04:16:09.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.558+0000 7fda90c00640 1 -- 172.21.15.3:0/3377631577 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fda8009bbf0 con 0x7fda941006a0 2024-08-01T04:16:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.562+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fda780a2b80 msgr2=0x7fda780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.562+0000 7fda99c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fda780a2b80 0x7fda780a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fda84009fc0 tx=0x7fda84005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.562+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 msgr2=0x7fda9419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:09.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.562+0000 7fda99c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 0x7fda9419a0d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000ee80 comp rx=0 tx=0).stop 2024-08-01T04:16:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.563+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 shutdown_connections 2024-08-01T04:16:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.563+0000 7fda99c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7fda780a2b80 0x7fda780a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.563+0000 7fda99c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda94101060 0x7fda941a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.563+0000 7fda99c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda941006a0 0x7fda9419a0d0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000ee80 comp rx=0 tx=0).stop 2024-08-01T04:16:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.563+0000 7fda99c00640 1 --2- 172.21.15.3:0/3377631577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda940ff940 0x7fda94199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:09.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.563+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 >> 172.21.15.3:0/3377631577 conn(0x7fda940fb480 msgr2=0x7fda940fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:09.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.565+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 shutdown_connections 2024-08-01T04:16:09.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:09.567+0000 7fda99c00640 1 -- 172.21.15.3:0/3377631577 wait complete. 2024-08-01T04:16:09.592 INFO:teuthology.run_tasks:Running task kclient... 2024-08-01T04:16:09.619 INFO:tasks.kclient:Mounting kernel clients... 2024-08-01T04:16:09.619 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-01T04:16:09.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:09.619 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-08-01T04:16:09.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:09.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-08-01T04:16:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-08-01T04:16:09.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:16:09.744 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-08-01T04:16:09.745 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-01T04:16:09.745 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:09.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs ls 2024-08-01T04:16:09.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:09 smithi003 ceph-mon[32262]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:16:09.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:09 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3377631577' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-01T04:16:09.802 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:09 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ignoring --setuser ceph since I am not root 2024-08-01T04:16:09.802 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:09 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ignoring --setgroup ceph since I am not root 2024-08-01T04:16:09.860 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-mon[37383]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:16:09.861 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3377631577' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-01T04:16:09.861 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: ignoring --setuser ceph since I am not root 2024-08-01T04:16:09.861 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: ignoring --setgroup ceph since I am not root 2024-08-01T04:16:09.861 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-mgr[39107]: -- 172.21.15.163:0/1991305335 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa66b3b0e0 con 0x55aa66b19400 2024-08-01T04:16:09.861 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:09.713+0000 7fc4b445d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:16:09.861 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:09.859+0000 7fc4b445d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:16:09.865 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-mon[37224]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:16:09.865 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3377631577' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-01T04:16:09.865 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: ignoring --setuser ceph since I am not root 2024-08-01T04:16:09.866 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: ignoring --setgroup ceph since I am not root 2024-08-01T04:16:09.866 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-mgr[38596]: -- 172.21.15.160:0/2877224844 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55778b2f70e0 con 0x55778b2d4800 2024-08-01T04:16:09.866 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:09.718+0000 7fc435854040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:16:09.866 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:09.863+0000 7fc435854040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:16:10.130 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:09 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:09.842+0000 7ff0b1e26040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-01T04:16:10.130 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:10 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:10.034+0000 7ff0b1e26040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:16:10.152 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:09.931+0000 7fc435854040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:16:10.163 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:09.926+0000 7fc4b445d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:16:10.163 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:10 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:10.162+0000 7fc4b445d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:16:10.206 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:10.384 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:10 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:10.127+0000 7ff0b1e26040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-01T04:16:10.545 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:10 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:10.221+0000 7fc4b445d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:16:10.562 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:10 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:10.171+0000 7fc435854040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:16:10.562 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:10 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:10.230+0000 7fc435854040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:16:10.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:10 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3377631577' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-01T04:16:10.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:10 smithi003 ceph-mon[32262]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-08-01T04:16:10.799 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:10 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:10.437+0000 7ff0b1e26040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-01T04:16:10.799 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:10 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:10.509+0000 7ff0b1e26040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-01T04:16:10.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:10 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3377631577' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-01T04:16:10.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:10 smithi160 ceph-mon[37224]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-08-01T04:16:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 -- 172.21.15.3:0/104947491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0102940 msgr2=0x7ff3a0102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:10.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 --2- 172.21.15.3:0/104947491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0102940 0x7ff3a0102d20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff390009900 tx=0x7ff39002a480 comp rx=0 tx=0).stop 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 -- 172.21.15.3:0/104947491 shutdown_connections 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 --2- 172.21.15.3:0/104947491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a010bd20 0x7ff3a010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 --2- 172.21.15.3:0/104947491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a0103260 0x7ff3a010b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 --2- 172.21.15.3:0/104947491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0102940 0x7ff3a0102d20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.939+0000 7ff39fe00640 1 -- 172.21.15.3:0/104947491 >> 172.21.15.3:0/104947491 conn(0x7ff3a00fc340 msgr2=0x7ff3a00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.940+0000 7ff39fe00640 1 -- 172.21.15.3:0/104947491 shutdown_connections 2024-08-01T04:16:10.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.940+0000 7ff39fe00640 1 -- 172.21.15.3:0/104947491 wait complete. 2024-08-01T04:16:10.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 Processor -- start 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 -- start start 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 0x7ff3a019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0103260 0x7ff3a019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a010bd20 0x7ff3a019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0111b40 con 0x7ff3a010bd20 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01119c0 con 0x7ff3a0102940 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a0111cc0 con 0x7ff3a0103260 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 0x7ff3a019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 0x7ff3a019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:55392/0 (socket says 172.21.15.3:55392) 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff39ea00640 1 -- 172.21.15.3:0/3557961805 learned_addr learned my addr 172.21.15.3:0/3557961805 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.941+0000 7ff397e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0103260 0x7ff3a019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:10.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39f400640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a010bd20 0x7ff3a019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:10.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39ea00640 1 -- 172.21.15.3:0/3557961805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0103260 msgr2=0x7ff3a019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39ea00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0103260 0x7ff3a019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39ea00640 1 -- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a010bd20 msgr2=0x7ff3a019ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39ea00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a010bd20 0x7ff3a019ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39ea00640 1 -- 172.21.15.3:0/3557961805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a019f170 con 0x7ff3a0102940 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff397e00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0103260 0x7ff3a019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39f400640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a010bd20 0x7ff3a019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.942+0000 7ff39ea00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 0x7ff3a019a1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff390004770 tx=0x7ff390003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.943+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39003f070 con 0x7ff3a0102940 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.943+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390031ca0 con 0x7ff3a0102940 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.943+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a019f400 con 0x7ff3a0102940 2024-08-01T04:16:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.943+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390035030 con 0x7ff3a0102940 2024-08-01T04:16:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.943+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a01a6d40 con 0x7ff3a0102940 2024-08-01T04:16:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.944+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a0102d20 con 0x7ff3a0102940 2024-08-01T04:16:10.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.946+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7ff390042070 con 0x7ff3a0102940 2024-08-01T04:16:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.947+0000 7ff397400640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 0x7ff3840a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.947+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8327+0+0 (secure 0 0 0) 0x7ff3900ef720 con 0x7ff3a0102940 2024-08-01T04:16:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.947+0000 7ff397e00640 1 -- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 msgr2=0x7ff3840a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1617846146 2024-08-01T04:16:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.947+0000 7ff397e00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 0x7ff3840a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-01T04:16:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:10.951+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff3900bc680 con 0x7ff3a0102940 2024-08-01T04:16:11.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:10 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3377631577' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-01T04:16:11.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:10 smithi163 ceph-mon[37383]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-08-01T04:16:11.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.148+0000 7ff397e00640 1 -- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 msgr2=0x7ff3840a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1617846146 2024-08-01T04:16:11.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.148+0000 7ff397e00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 0x7ff3840a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-01T04:16:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.180+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3a01a6f10 con 0x7ff3a0102940 2024-08-01T04:16:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.181+0000 7ff397400640 1 -- 172.21.15.3:0/3557961805 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7ff3900bbf80 con 0x7ff3a0102940 2024-08-01T04:16:11.183 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-01T04:16:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 msgr2=0x7ff3840a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:16:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 0x7ff3840a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 msgr2=0x7ff3a019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 0x7ff3a019a1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff390004770 tx=0x7ff390003c10 comp rx=0 tx=0).stop 2024-08-01T04:16:11.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 shutdown_connections 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:6800/1617846146,v1:172.21.15.3:6801/1617846146] conn(0x7ff3840a2c50 0x7ff3840a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3a010bd20 0x7ff3a019ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a0103260 0x7ff3a019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 --2- 172.21.15.3:0/3557961805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff3a0102940 0x7ff3a019a1c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.184+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 >> 172.21.15.3:0/3557961805 conn(0x7ff3a00fc340 msgr2=0x7ff3a00fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.185+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 shutdown_connections 2024-08-01T04:16:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:11.185+0000 7ff39fe00640 1 -- 172.21.15.3:0/3557961805 wait complete. 2024-08-01T04:16:11.522 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.143+0000 7fc4b445d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:16:11.522 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.256+0000 7fc4b445d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:16:11.522 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.312+0000 7fc4b445d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:16:11.522 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.520+0000 7fc4b445d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:16:11.544 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.161+0000 7fc435854040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:16:11.544 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.275+0000 7fc435854040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:16:11.545 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.332+0000 7fc435854040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:16:11.545 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.543+0000 7fc435854040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:16:11.793 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3557961805' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-01T04:16:11.794 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.582+0000 7fc4b445d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:16:11.794 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.648+0000 7fc4b445d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:16:11.817 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3557961805' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-01T04:16:11.818 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.605+0000 7fc435854040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:16:11.818 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.673+0000 7fc435854040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:16:11.818 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:11.816+0000 7fc435854040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:16:11.880 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:11 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3557961805' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-01T04:16:11.880 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:11.771+0000 7ff0b1e26040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-01T04:16:12.045 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:11.792+0000 7fc4b445d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:16:12.241 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:11.915+0000 7ff0b1e26040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-01T04:16:12.241 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:11 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:11.985+0000 7ff0b1e26040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-01T04:16:12.321 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-01T04:16:12.322 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-01T04:16:12.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:12.322 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-08-01T04:16:12.374 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-01T04:16:12.374 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-01T04:16:12.374 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-08-01T04:16:12.374 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-08-01T04:16:12.374 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-08-01T04:16:12.374 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41978sec preferred_lft 41978sec 2024-08-01T04:16:12.375 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-08-01T04:16:12.376 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-08-01T04:16:12.376 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-01T04:16:12.376 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-08-01T04:16:12.376 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-08-01T04:16:12.377 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-01T04:16:12.377 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-01T04:16:12.377 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-08-01T04:16:12.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-08-01T04:16:12.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-08-01T04:16:12.377 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-08-01T04:16:12.378 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-01T04:16:12.378 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-08-01T04:16:12.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-01T04:16:12.499 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:12 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:12.244+0000 7ff0b1e26040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-01T04:16:12.499 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:12 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:12.321+0000 7ff0b1e26040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-01T04:16:12.500 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:12 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:12.405+0000 7ff0b1e26040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-01T04:16:12.518 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:12.205+0000 7fc4b445d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:16:12.547 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:12.233+0000 7fc435854040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:16:12.548 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:12.546+0000 7fc435854040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:16:12.775 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:12 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:12.584+0000 7ff0b1e26040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-01T04:16:12.795 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:12.517+0000 7fc4b445d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:16:12.795 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:12.582+0000 7fc4b445d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:16:12.795 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:12.732+0000 7fc4b445d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:16:12.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:12.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:12.886 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-01T04:16:12.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:12.612+0000 7fc435854040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:16:12.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:12.762+0000 7fc435854040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:16:12.950 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-08-01T04:16:12.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:12.954 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-08-01T04:16:13.052 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-08-01T04:16:13.053 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-01T04:16:13.053 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-01T04:16:13.053 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-01T04:16:13.053 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-01T04:16:13.054 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-01T04:16:13.054 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-08-01T04:16:13.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-01T04:16:13.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-01T04:16:13.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-01T04:16:13.054 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-08-01T04:16:13.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-01T04:16:13.295 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.040+0000 7fc4b445d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:16:13.296 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.100+0000 7fc4b445d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:16:13.296 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.281+0000 7fc4b445d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:16:13.402 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.071+0000 7fc435854040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:16:13.402 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.132+0000 7fc435854040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:16:13.402 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.315+0000 7fc435854040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:16:13.510 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:13.148+0000 7ff0b1e26040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-01T04:16:13.795 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.426+0000 7fc4b445d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:16:13.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.489+0000 7fc4b445d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:16:13.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.571+0000 7fc4b445d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:16:13.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:13.630+0000 7fc4b445d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:16:13.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:13.531+0000 7ff0b1e26040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-01T04:16:13.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:13.616+0000 7ff0b1e26040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-01T04:16:13.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.461+0000 7fc435854040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:16:13.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.525+0000 7fc435854040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:16:13.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.607+0000 7fc435854040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:16:13.902 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:13 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:13.666+0000 7fc435854040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:16:14.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:14.050 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-08-01T04:16:14.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:14.097 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-08-01T04:16:14.187 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:13 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:13.799+0000 7ff0b1e26040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-01T04:16:14.191 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-01T04:16:14.191 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-08-01T04:16:14.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-01T04:16:14.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-01T04:16:14.191 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-08-01T04:16:14.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-01T04:16:14.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:14.371 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-01T04:16:14.372 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-08-01T04:16:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-01T04:16:14.485 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.185+0000 7ff0b1e26040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-01T04:16:14.485 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.263+0000 7ff0b1e26040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: Standby manager daemon z restarted 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: Standby manager daemon z started 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: Standby manager daemon y restarted 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: Standby manager daemon y started 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:16:14.486 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-01T04:16:14.487 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-mon[32262]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:14.160+0000 7fc4b445d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z[39103]: 2024-08-01T04:16:14.318+0000 7fc4b445d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: Standby manager daemon z restarted 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: Standby manager daemon z started 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-01T04:16:14.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:16:14.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: Standby manager daemon y restarted 2024-08-01T04:16:14.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: Standby manager daemon y started 2024-08-01T04:16:14.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-01T04:16:14.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:16:14.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-01T04:16:14.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:14 smithi163 ceph-mon[37383]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:16:14.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:14.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:14.202+0000 7fc435854040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:16:14.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y[38592]: 2024-08-01T04:16:14.361+0000 7fc435854040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:16:14.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: Standby manager daemon z restarted 2024-08-01T04:16:14.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: Standby manager daemon z started 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.163:0/4192159354' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: Standby manager daemon y restarted 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: Standby manager daemon y started 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-01T04:16:14.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:14 smithi160 ceph-mon[37224]: from='mgr.? 172.21.15.160:0/2206032113' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-01T04:16:14.657 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-01T04:16:14.657 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-08-01T04:16:14.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-08-01T04:16:14.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-08-01T04:16:14.657 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-08-01T04:16:14.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-01T04:16:14.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.505+0000 7ff0b1e26040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-01T04:16:14.771 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.707+0000 7ff0b1e26040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-01T04:16:14.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:14.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-01T04:16:14.854 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:14.895 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-01T04:16:14.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-01T04:16:14.895 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:14.986 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-01T04:16:14.986 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-01T04:16:14.986 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/0d0a8131-aa87-46a5-b639-6265a67bbe44 2024-08-01T04:16:14.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:14.987 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/0d0a8131-aa87-46a5-b639-6265a67bbe44 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-08-01T04:16:15.045 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.790+0000 7ff0b1e26040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-01T04:16:15.045 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.902+0000 7ff0b1e26040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-01T04:16:15.045 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:14 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:14.974+0000 7ff0b1e26040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-01T04:16:15.126 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-08-01T04:16:15.127 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-08-01T04:16:15.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:15 smithi003 ceph-mon[32262]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-08-01T04:16:15.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:15 smithi160 ceph-mon[37224]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-08-01T04:16:15.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:15 smithi163 ceph-mon[37383]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-08-01T04:16:15.993 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:15.633+0000 7ff0b1e26040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-01T04:16:15.993 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:15.824+0000 7ff0b1e26040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-01T04:16:16.011 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-08-01T04:16:16.011 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-08-01T04:16:16.011 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.160:6789,172.21.15.163:6789" 2024-08-01T04:16:16.011 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@2f6d75a6-4fbb-11ef-bcc9-c7b262605968.cephfs=/volumes/qa/sv_1/0d0a8131-aa87-46a5-b639-6265a67bbe44 2024-08-01T04:16:16.012 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.3:6789/172.21.15.160:6789/172.21.15.163:6789" will pass to kernel 2024-08-01T04:16:16.014 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-01T04:16:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-08-01T04:16:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-01T04:16:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-08-01T04:16:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-01T04:16:16.020 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-01T04:16:16.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:16:16.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:16.128 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-01T04:16:16.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-01T04:16:16.252 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.248+0000 7ff00b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.252 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.248+0000 7ff00b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.248+0000 7ff00b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.248+0000 7ff00b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.248+0000 7ff00b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.250+0000 7ff00e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.250+0000 7ff00e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.250+0000 7ff00e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.250+0000 7ff00e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.253 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:16.250+0000 7ff00e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:16.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:16.267 INFO:teuthology.orchestra.run.smithi003.stdout:client25091 2024-08-01T04:16:16.267 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:16:16.267 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-01T04:16:16.308 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-01T04:16:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-01T04:16:16.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:16.455 INFO:teuthology.orchestra.run.smithi003.stdout:client25091 2024-08-01T04:16:16.455 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-01T04:16:16.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: Active manager daemon x restarted 2024-08-01T04:16:16.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: Activating manager daemon x 2024-08-01T04:16:16.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: osdmap e91: 12 total, 12 up, 12 in 2024-08-01T04:16:16.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: mgrmap e21: x(active, starting, since 0.0233944s), standbys: z, y 2024-08-01T04:16:16.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-01T04:16:16.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:16:16.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-01T04:16:16.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: Manager daemon x is now available 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:16:16.553 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:16 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:16:16.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-01T04:16:16.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:16.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: Active manager daemon x restarted 2024-08-01T04:16:16.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: Activating manager daemon x 2024-08-01T04:16:16.568 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: osdmap e91: 12 total, 12 up, 12 in 2024-08-01T04:16:16.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: mgrmap e21: x(active, starting, since 0.0233944s), standbys: z, y 2024-08-01T04:16:16.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:16:16.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:16:16.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:16:16.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-01T04:16:16.569 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-01T04:16:16.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-01T04:16:16.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-01T04:16:16.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-01T04:16:16.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-01T04:16:16.570 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:16:16.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:16:16.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:16:16.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:16:16.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:16:16.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:16:16.571 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:16:16.572 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:16:16.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:16:16.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:16:16.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:16:16.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:16:16.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:16:16.573 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:16:16.574 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:16:16.574 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-01T04:16:16.574 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-01T04:16:16.574 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-01T04:16:16.574 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: Manager daemon x is now available 2024-08-01T04:16:16.575 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:16.575 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:16:16.575 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:16:16.575 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:16:16.575 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:16 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:16:16.577 INFO:teuthology.orchestra.run.smithi003.stdout:client25091 2024-08-01T04:16:16.577 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968.client25091/status') 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: Active manager daemon x restarted 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: Activating manager daemon x 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: osdmap e91: 12 total, 12 up, 12 in 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: mgrmap e21: x(active, starting, since 0.0233944s), standbys: z, y 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-01T04:16:16.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:16:16.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-01T04:16:16.655 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: Manager daemon x is now available 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:16:16.656 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-01T04:16:16.657 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:16:16.657 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:16 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-01T04:16:16.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-01T04:16:16.698 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.25091 (3)172.21.15.3:0/3884076632 2024-08-01T04:16:16.698 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-08-01T04:16:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-08-01T04:16:16.698 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-08-01T04:16:16.699 INFO:teuthology.orchestra.run.smithi003.stderr:70 bytes copied, 6.2011e-05 s, 1.1 MB/s 2024-08-01T04:16:16.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-01T04:16:16.711 INFO:teuthology.run_tasks:Running task exec... 2024-08-01T04:16:16.721 INFO:teuthology.task.exec:Executing custom commands... 2024-08-01T04:16:16.721 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:16:16.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 -- 172.21.15.3:0/212913663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c1012b0 msgr2=0x7f587c101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 --2- 172.21.15.3:0/212913663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c1012b0 0x7f587c101730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5868009900 tx=0x7f586802a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 -- 172.21.15.3:0/212913663 shutdown_connections 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 --2- 172.21.15.3:0/212913663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c101c70 0x7f587c109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 --2- 172.21.15.3:0/212913663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c1012b0 0x7f587c101730 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 --2- 172.21.15.3:0/212913663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c102d80 0x7f587c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 -- 172.21.15.3:0/212913663 >> 172.21.15.3:0/212913663 conn(0x7f587c0fce00 msgr2=0x7f587c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 -- 172.21.15.3:0/212913663 shutdown_connections 2024-08-01T04:16:16.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.899+0000 7f5881c00640 1 -- 172.21.15.3:0/212913663 wait complete. 2024-08-01T04:16:16.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.900+0000 7f5881c00640 1 Processor -- start 2024-08-01T04:16:16.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.900+0000 7f5881c00640 1 -- start start 2024-08-01T04:16:16.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 0x7f587c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:16.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c101c70 0x7f587c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:16.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c102d80 0x7f587c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:16.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d400 con 0x7f587c101c70 2024-08-01T04:16:16.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d280 con 0x7f587c1012b0 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10d580 con 0x7f587c102d80 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 0x7f587c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 0x7f587c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33262/0 (socket says 172.21.15.3:33262) 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f5881200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c102d80 0x7f587c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 -- 172.21.15.3:0/376313023 learned_addr learned my addr 172.21.15.3:0/376313023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587b400640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c101c70 0x7f587c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 -- 172.21.15.3:0/376313023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c102d80 msgr2=0x7f587c1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c102d80 0x7f587c1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 -- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c101c70 msgr2=0x7f587c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.901+0000 7f587be00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c101c70 0x7f587c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:16.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f587be00640 1 -- 172.21.15.3:0/376313023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c1a1960 con 0x7f587c1012b0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f587be00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 0x7f587c199d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e9c0 tx=0x7f586c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00cd80 con 0x7f587c1012b0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c1a1c50 con 0x7f587c1012b0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586c010910 con 0x7f587c1012b0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c018980 con 0x7f587c1012b0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.902+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c1a2170 con 0x7f587c1012b0 2024-08-01T04:16:16.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.903+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c101730 con 0x7f587c1012b0 2024-08-01T04:16:16.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.905+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f586c01f020 con 0x7f587c1012b0 2024-08-01T04:16:16.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.906+0000 7f5878c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f58600a2970 0x7f58600a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:16.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.907+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f586c0cef40 con 0x7f587c1012b0 2024-08-01T04:16:16.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.907+0000 7f587b400640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f58600a2970 0x7f58600a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:16.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.908+0000 7f587b400640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f58600a2970 0x7f58600a4e30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5868000c00 tx=0x7f58680315d0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:16.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:16.910+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f586c09bff0 con 0x7f587c1012b0 2024-08-01T04:16:17.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.176+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f587c1a2a10 con 0x7f587c1012b0 2024-08-01T04:16:17.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.872+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f586c097f60 con 0x7f587c1012b0 2024-08-01T04:16:17.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.874+0000 7f5878c00640 1 -- 172.21.15.3:0/376313023 <== mon.2 v2:172.21.15.160:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f586c077ce0 con 0x7f587c1012b0 2024-08-01T04:16:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.877+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f58600a2970 msgr2=0x7f58600a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.877+0000 7f5881c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f58600a2970 0x7f58600a4e30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5868000c00 tx=0x7f58680315d0 comp rx=0 tx=0).stop 2024-08-01T04:16:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.877+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 msgr2=0x7f587c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:17.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.877+0000 7f5881c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 0x7f587c199d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e9c0 tx=0x7f586c00edc0 comp rx=0 tx=0).stop 2024-08-01T04:16:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.878+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 shutdown_connections 2024-08-01T04:16:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.878+0000 7f5881c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f58600a2970 0x7f58600a4e30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.878+0000 7f5881c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f587c102d80 0x7f587c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.878+0000 7f5881c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c101c70 0x7f587c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.878+0000 7f5881c00640 1 --2- 172.21.15.3:0/376313023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f587c1012b0 0x7f587c199d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:17.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.878+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 >> 172.21.15.3:0/376313023 conn(0x7f587c0fce00 msgr2=0x7f587c108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.879+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 shutdown_connections 2024-08-01T04:16:17.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:17.879+0000 7f5881c00640 1 -- 172.21.15.3:0/376313023 wait complete. 2024-08-01T04:16:17.898 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-08-01T04:16:18.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:17 smithi003 ceph-mon[32262]: mgrmap e22: x(active, since 1.02339s), standbys: z, y 2024-08-01T04:16:18.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:17 smithi003 ceph-mon[32262]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:18.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:17 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/376313023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-01T04:16:18.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:17 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-01T04:16:18.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 -- 172.21.15.3:0/373294851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 msgr2=0x7f1748109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 --2- 172.21.15.3:0/373294851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748109ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1740009ca0 tx=0x7f174002a8b0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 -- 172.21.15.3:0/373294851 shutdown_connections 2024-08-01T04:16:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 --2- 172.21.15.3:0/373294851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748109ae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 --2- 172.21.15.3:0/373294851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 0x7f1748101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 --2- 172.21.15.3:0/373294851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1748102d80 0x7f1748103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 -- 172.21.15.3:0/373294851 >> 172.21.15.3:0/373294851 conn(0x7f17480fce00 msgr2=0x7f17480ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 -- 172.21.15.3:0/373294851 shutdown_connections 2024-08-01T04:16:18.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.077+0000 7f174e200640 1 -- 172.21.15.3:0/373294851 wait complete. 2024-08-01T04:16:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.078+0000 7f174e200640 1 Processor -- start 2024-08-01T04:16:18.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 -- start start 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 0x7f1748197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1748102d80 0x7f174819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f174810d470 con 0x7f1748101c70 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f174810d2f0 con 0x7f17481012b0 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174e200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f174810d5f0 con 0x7f1748102d80 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f1747e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174ce00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 0x7f1748197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f1747e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53354/0 (socket says 172.21.15.3:53354) 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f1747e00640 1 -- 172.21.15.3:0/104616846 learned_addr learned my addr 172.21.15.3:0/104616846 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.079+0000 7f174d800640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1748102d80 0x7f174819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174ce00640 1 -- 172.21.15.3:0/104616846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1748102d80 msgr2=0x7f174819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174ce00640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1748102d80 0x7f174819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174ce00640 1 -- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 msgr2=0x7f1748197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174ce00640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174ce00640 1 -- 172.21.15.3:0/104616846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174819f680 con 0x7f17481012b0 2024-08-01T04:16:18.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174ce00640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 0x7f1748197a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e9c0 tx=0x7f173c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c00ccc0 con 0x7f17481012b0 2024-08-01T04:16:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f173c0035f0 con 0x7f17481012b0 2024-08-01T04:16:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c00ce60 con 0x7f17481012b0 2024-08-01T04:16:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174819f970 con 0x7f17481012b0 2024-08-01T04:16:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.080+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174819fec0 con 0x7f17481012b0 2024-08-01T04:16:18.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.081+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1748101730 con 0x7f17481012b0 2024-08-01T04:16:18.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.083+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f173c0037a0 con 0x7f17481012b0 2024-08-01T04:16:18.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.085+0000 7f1745600640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f17300a2d70 0x7f17300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.085+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f173c0cf970 con 0x7f17481012b0 2024-08-01T04:16:18.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.085+0000 7f1747e00640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f17300a2d70 0x7f17300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.086+0000 7f1747e00640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f17300a2d70 0x7f17300a5230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1748198f70 tx=0x7f1734005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:18.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.088+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f173c09c4f0 con 0x7f17481012b0 2024-08-01T04:16:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:17 smithi160 ceph-mon[37224]: mgrmap e22: x(active, since 1.02339s), standbys: z, y 2024-08-01T04:16:18.220 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:17 smithi160 ceph-mon[37224]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:18.220 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:17 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/376313023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-01T04:16:18.220 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:17 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-01T04:16:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:17 smithi163 ceph-mon[37383]: mgrmap e22: x(active, since 1.02339s), standbys: z, y 2024-08-01T04:16:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:17 smithi163 ceph-mon[37383]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:17 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/376313023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-01T04:16:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:17 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-01T04:16:18.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.360+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1748062400 con 0x7f17300a2d70 2024-08-01T04:16:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.381+0000 7f1745600640 1 -- 172.21.15.3:0/104616846 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1748062400 con 0x7f17300a2d70 2024-08-01T04:16:18.383 INFO:teuthology.orchestra.run.smithi003.stdout:{} 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f17300a2d70 msgr2=0x7f17300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f17300a2d70 0x7f17300a5230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1748198f70 tx=0x7f1734005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 msgr2=0x7f1748197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 0x7f1748197a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f173c00e9c0 tx=0x7f173c00ee80 comp rx=0 tx=0).stop 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 shutdown_connections 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f17300a2d70 0x7f17300a5230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1748102d80 0x7f174819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1748101c70 0x7f1748197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 --2- 172.21.15.3:0/104616846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17481012b0 0x7f1748197a60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 >> 172.21.15.3:0/104616846 conn(0x7f17480fce00 msgr2=0x7f1748105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 shutdown_connections 2024-08-01T04:16:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.384+0000 7f174e200640 1 -- 172.21.15.3:0/104616846 wait complete. 2024-08-01T04:16:18.402 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-01T04:16:18.413 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-01T04:16:18.444 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-01T04:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 -- 172.21.15.3:0/918840042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 msgr2=0x7fcae4074de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 --2- 172.21.15.3:0/918840042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae4074de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009900 tx=0x7fcacc02a410 comp rx=0 tx=0).stop 2024-08-01T04:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 -- 172.21.15.3:0/918840042 shutdown_connections 2024-08-01T04:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 --2- 172.21.15.3:0/918840042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae410de30 0x7fcae4110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 --2- 172.21.15.3:0/918840042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae4074de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 --2- 172.21.15.3:0/918840042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae40743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 -- 172.21.15.3:0/918840042 >> 172.21.15.3:0/918840042 conn(0x7fcae40fcfe0 msgr2=0x7fcae40ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 -- 172.21.15.3:0/918840042 shutdown_connections 2024-08-01T04:16:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.636+0000 7fcae9a00640 1 -- 172.21.15.3:0/918840042 wait complete. 2024-08-01T04:16:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.637+0000 7fcae9a00640 1 Processor -- start 2024-08-01T04:16:18.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 -- start start 2024-08-01T04:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae410de30 0x7fcae41a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4113b00 con 0x7fcae410de30 2024-08-01T04:16:18.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4113980 con 0x7fcae4074980 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae9a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4113c80 con 0x7fcae4075f60 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcada000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcae2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcada000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53298/0 (socket says 172.21.15.3:53298) 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.638+0000 7fcada000640 1 -- 172.21.15.3:0/1389482119 learned_addr learned my addr 172.21.15.3:0/1389482119 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcae3400640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae410de30 0x7fcae41a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcada000640 1 -- 172.21.15.3:0/1389482119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 msgr2=0x7fcae419e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcada000640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae419e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcada000640 1 -- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae410de30 msgr2=0x7fcae41a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcada000640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae410de30 0x7fcae41a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcada000640 1 -- 172.21.15.3:0/1389482119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae41a5de0 con 0x7fcae4075f60 2024-08-01T04:16:18.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcae2a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcada000640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae419e700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcacc031440 tx=0x7fcacc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc03f030 con 0x7fcae4075f60 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.639+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcacc003ce0 con 0x7fcae4075f60 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.640+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc035070 con 0x7fcae4075f60 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.640+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae41a6070 con 0x7fcae4075f60 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.640+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae41a6560 con 0x7fcae4075f60 2024-08-01T04:16:18.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.641+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae4104000 con 0x7fcae4075f60 2024-08-01T04:16:18.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.643+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7fcacc042030 con 0x7fcae4075f60 2024-08-01T04:16:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.645+0000 7fcadbe00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcac80a2d00 0x7fcac80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.645+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fcacc0ef970 con 0x7fcae4075f60 2024-08-01T04:16:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.645+0000 7fcae2a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcac80a2d00 0x7fcac80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:18.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.646+0000 7fcae2a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcac80a2d00 0x7fcac80a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009fc0 tx=0x7fcad4005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:18.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.648+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcacc0bbea0 con 0x7fcae4075f60 2024-08-01T04:16:18.799 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.367+0000 7ff007a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.367+0000 7ff007a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.367+0000 7ff007a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.367+0000 7ff007a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.367+0000 7ff007a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.378+0000 7fefee000640 -1 log_channel(cephadm) log [ERR] : [01/Aug/2024:04:16:18] ENGINE Error in HTTPServer.serve 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: Traceback (most recent call last): 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._connections.run(self.expiration_interval) 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._run(expiration_interval) 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:16:18.800 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: s = self.context.wrap_socket( 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: return self.sslsocket_class._create( 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self.do_handshake() 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: self._sslobj.do_handshake() 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:16:18.801 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:16:18.874 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-01T04:16:18.874 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:16:18.874 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-08-01T04:16:18.874 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.874 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: [01/Aug/2024:04:16:18] ENGINE Bus STARTING 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: [01/Aug/2024:04:16:18] ENGINE Serving on https://172.21.15.3:7150 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: [01/Aug/2024:04:16:18] ENGINE Error in HTTPServer.serve 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:16:18.875 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: [01/Aug/2024:04:16:18] ENGINE Serving on http://172.21.15.3:8765 2024-08-01T04:16:18.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: [01/Aug/2024:04:16:18] ENGINE Bus STARTED 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:18.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.880+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcae402d830 con 0x7fcae4075f60 2024-08-01T04:16:18.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.881+0000 7fcadbe00640 1 -- 172.21.15.3:0/1389482119 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23178 (secure 0 0 0) 0x7fcacc0c01a0 con 0x7fcae4075f60 2024-08-01T04:16:18.884 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:18.884 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:16:15.828445+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1348130449":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/346159818":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/4015366337":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2309230633":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcac80a2d00 msgr2=0x7fcac80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcac80a2d00 0x7fcac80a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009fc0 tx=0x7fcad4005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 msgr2=0x7fcae419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae419e700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcacc031440 tx=0x7fcacc032ce0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 shutdown_connections 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae410de30 0x7fcae41a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcac80a2d00 0x7fcac80a51c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae4075f60 0x7fcae419e700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 --2- 172.21.15.3:0/1389482119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4074980 0x7fcae419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 >> 172.21.15.3:0/1389482119 conn(0x7fcae40fcfe0 msgr2=0x7fcae41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 shutdown_connections 2024-08-01T04:16:18.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:18.884+0000 7fcae9a00640 1 -- 172.21.15.3:0/1389482119 wait complete. 2024-08-01T04:16:18.904 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-01T04:09:04.723203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-01T04:13:50.125660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-01T04:13:53.246715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-01T04:16:18.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-01T04:16:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 -- 172.21.15.3:0/1825568682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658102910 msgr2=0x7f2658102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 --2- 172.21.15.3:0/1825568682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658102910 0x7f2658102d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2640009900 tx=0x7f264002a400 comp rx=0 tx=0).stop 2024-08-01T04:16:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 -- 172.21.15.3:0/1825568682 shutdown_connections 2024-08-01T04:16:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 --2- 172.21.15.3:0/1825568682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26581032d0 0x7f2658109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 --2- 172.21.15.3:0/1825568682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658102910 0x7f2658102d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 --2- 172.21.15.3:0/1825568682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2658101720 0x7f2658101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 -- 172.21.15.3:0/1825568682 >> 172.21.15.3:0/1825568682 conn(0x7f26580fcec0 msgr2=0x7f26580ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.085+0000 7f265d800640 1 -- 172.21.15.3:0/1825568682 shutdown_connections 2024-08-01T04:16:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.086+0000 7f265d800640 1 -- 172.21.15.3:0/1825568682 wait complete. 2024-08-01T04:16:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 Processor -- start 2024-08-01T04:16:19.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 -- start start 2024-08-01T04:16:19.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 0x7f2658199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2658102910 0x7f265819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26581032d0 0x7f26581a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d380 con 0x7f26581032d0 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d200 con 0x7f2658102910 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f265d800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f265810d500 con 0x7f2658101720 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f2657e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 0x7f2658199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f2657e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 0x7f2658199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53326/0 (socket says 172.21.15.3:53326) 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.087+0000 7f2657e00640 1 -- 172.21.15.3:0/3951613901 learned_addr learned my addr 172.21.15.3:0/3951613901 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f264fe00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2658102910 0x7f265819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f265ce00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26581032d0 0x7f26581a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f2657e00640 1 -- 172.21.15.3:0/3951613901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2658102910 msgr2=0x7f265819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f2657e00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2658102910 0x7f265819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f2657e00640 1 -- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26581032d0 msgr2=0x7f26581a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f2657e00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26581032d0 0x7f26581a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f2657e00640 1 -- 172.21.15.3:0/3951613901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581a1860 con 0x7f2658101720 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f265ce00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26581032d0 0x7f26581a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.088+0000 7f2657e00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 0x7f2658199c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f264800e9c0 tx=0x7f264800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.089+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264800cd80 con 0x7f2658101720 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.089+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26480035f0 con 0x7f2658101720 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.089+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581a1b50 con 0x7f2658101720 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.089+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648010690 con 0x7f2658101720 2024-08-01T04:16:19.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.089+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a20a0 con 0x7f2658101720 2024-08-01T04:16:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.090+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26581a2270 con 0x7f2658101720 2024-08-01T04:16:19.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.091+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f26480037a0 con 0x7f2658101720 2024-08-01T04:16:19.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.093+0000 7f2655600640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f263c0a2d00 0x7f263c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.093+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f26480cf9e0 con 0x7f2658101720 2024-08-01T04:16:19.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.093+0000 7f264fe00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f263c0a2d00 0x7f263c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.094+0000 7f264fe00640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f263c0a2d00 0x7f263c0a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26400314e0 tx=0x7f2640000bc0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:19.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.096+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f264809c560 con 0x7f2658101720 2024-08-01T04:16:19.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: [01/Aug/2024:04:16:18] ENGINE Bus STARTING 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: [01/Aug/2024:04:16:18] ENGINE Serving on https://172.21.15.3:7150 2024-08-01T04:16:19.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: [01/Aug/2024:04:16:18] ENGINE Error in HTTPServer.serve 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: Traceback (most recent call last): 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: self._connections.run(self.expiration_interval) 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: self._run(expiration_interval) 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: s = self.context.wrap_socket( 2024-08-01T04:16:19.154 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout: return self.sslsocket_class._create( 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout: self.do_handshake() 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout: self._sslobj.do_handshake() 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: [01/Aug/2024:04:16:18] ENGINE Serving on http://172.21.15.3:8765 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: [01/Aug/2024:04:16:18] ENGINE Bus STARTED 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.156 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.157 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.157 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.157 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.157 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:18 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-01T04:16:19.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:16:19.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-08-01T04:16:19.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: [01/Aug/2024:04:16:18] ENGINE Bus STARTING 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: [01/Aug/2024:04:16:18] ENGINE Serving on https://172.21.15.3:7150 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: [01/Aug/2024:04:16:18] ENGINE Error in HTTPServer.serve 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout: Traceback (most recent call last): 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout: self._connections.run(self.expiration_interval) 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-01T04:16:19.297 INFO:journalctl@ceph.mon.c.smithi163.stdout: self._run(expiration_interval) 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: s = self.context.wrap_socket( 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: return self.sslsocket_class._create( 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: self.do_handshake() 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: self._sslobj.do_handshake() 2024-08-01T04:16:19.298 INFO:journalctl@ceph.mon.c.smithi163.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: [01/Aug/2024:04:16:18] ENGINE Serving on http://172.21.15.3:8765 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: [01/Aug/2024:04:16:18] ENGINE Bus STARTED 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.299 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.300 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:18 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:18 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:19.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.312+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26581a2440 con 0x7f2658101720 2024-08-01T04:16:19.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.312+0000 7f2655600640 1 -- 172.21.15.3:0/3951613901 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f264809be60 con 0x7f2658101720 2024-08-01T04:16:19.314 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-08-01T04:16:19.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.314+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f263c0a2d00 msgr2=0x7f263c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.314+0000 7f265d800640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f263c0a2d00 0x7f263c0a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26400314e0 tx=0x7f2640000bc0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 msgr2=0x7f2658199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 0x7f2658199c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f264800e9c0 tx=0x7f264800ee80 comp rx=0 tx=0).stop 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 shutdown_connections 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26581032d0 0x7f26581a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2658102910 0x7f265819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f263c0a2d00 0x7f263c0a51c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 --2- 172.21.15.3:0/3951613901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2658101720 0x7f2658199c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 >> 172.21.15.3:0/3951613901 conn(0x7f26580fcec0 msgr2=0x7f26581080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 shutdown_connections 2024-08-01T04:16:19.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.315+0000 7f265d800640 1 -- 172.21.15.3:0/3951613901 wait complete. 2024-08-01T04:16:19.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-01T04:16:19.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 -- 172.21.15.3:0/700777352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 msgr2=0x7f1848101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 --2- 172.21.15.3:0/700777352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848101b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f183c009900 tx=0x7f183c02a410 comp rx=0 tx=0).stop 2024-08-01T04:16:19.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 -- 172.21.15.3:0/700777352 shutdown_connections 2024-08-01T04:16:19.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 --2- 172.21.15.3:0/700777352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18481032e0 0x7f1848109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 --2- 172.21.15.3:0/700777352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1848102920 0x7f1848102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 --2- 172.21.15.3:0/700777352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848101b30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 -- 172.21.15.3:0/700777352 >> 172.21.15.3:0/700777352 conn(0x7f18480fceb0 msgr2=0x7f18480ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 -- 172.21.15.3:0/700777352 shutdown_connections 2024-08-01T04:16:19.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.520+0000 7f184f200640 1 -- 172.21.15.3:0/700777352 wait complete. 2024-08-01T04:16:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.521+0000 7f184f200640 1 Processor -- start 2024-08-01T04:16:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.521+0000 7f184f200640 1 -- start start 2024-08-01T04:16:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848075cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1848102920 0x7f1848074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18481032e0 0x7f1848074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f184810d390 con 0x7f1848102920 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f184810d210 con 0x7f18481032e0 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f184810d510 con 0x7f1848101730 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1848102920 0x7f1848074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848075cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848075cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53338/0 (socket says 172.21.15.3:53338) 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184de00640 1 -- 172.21.15.3:0/716683090 learned_addr learned my addr 172.21.15.3:0/716683090 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.522+0000 7f184e800640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18481032e0 0x7f1848074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184de00640 1 -- 172.21.15.3:0/716683090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18481032e0 msgr2=0x7f1848074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184de00640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18481032e0 0x7f1848074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184de00640 1 -- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1848102920 msgr2=0x7f1848074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184de00640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1848102920 0x7f1848074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184de00640 1 -- 172.21.15.3:0/716683090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18480750e0 con 0x7f1848101730 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184e800640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18481032e0 0x7f1848074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184de00640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848075cd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f183c031610 tx=0x7f183c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c03f030 con 0x7f1848101730 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481a1b00 con 0x7f1848101730 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c003ad0 con 0x7f1848101730 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c035070 con 0x7f1848101730 2024-08-01T04:16:19.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.523+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a20a0 con 0x7f1848101730 2024-08-01T04:16:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.525+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1848108b20 con 0x7f1848101730 2024-08-01T04:16:19.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.526+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f183c042030 con 0x7f1848101730 2024-08-01T04:16:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.527+0000 7f1842a00640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18300a2d00 0x7f18300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.528+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f183c0efb00 con 0x7f1848101730 2024-08-01T04:16:19.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.528+0000 7f184d400640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18300a2d00 0x7f18300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.529+0000 7f184d400640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18300a2d00 0x7f18300a51c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1848075a40 tx=0x7f1838005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:19.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.531+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f183c0bc680 con 0x7f1848101730 2024-08-01T04:16:19.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.736+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f184802d830 con 0x7f1848101730 2024-08-01T04:16:19.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.737+0000 7f1842a00640 1 -- 172.21.15.3:0/716683090 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f183c0bbf80 con 0x7f1848101730 2024-08-01T04:16:19.739 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-08-01T04:16:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.739+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18300a2d00 msgr2=0x7f18300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.739+0000 7f184f200640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18300a2d00 0x7f18300a51c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1848075a40 tx=0x7f1838005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.739+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 msgr2=0x7f1848075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848075cd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f183c031610 tx=0x7f183c004270 comp rx=0 tx=0).stop 2024-08-01T04:16:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 shutdown_connections 2024-08-01T04:16:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18481032e0 0x7f1848074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1848102920 0x7f1848074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18300a2d00 0x7f18300a51c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 --2- 172.21.15.3:0/716683090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1848101730 0x7f1848075cd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 >> 172.21.15.3:0/716683090 conn(0x7f18480fceb0 msgr2=0x7f1848105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 shutdown_connections 2024-08-01T04:16:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.740+0000 7f184f200640 1 -- 172.21.15.3:0/716683090 wait complete. 2024-08-01T04:16:19.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-01T04:16:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 -- 172.21.15.3:0/3570975716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 msgr2=0x7fa59c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 --2- 172.21.15.3:0/3570975716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 0x7fa59c104750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009900 tx=0x7fa58c02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 -- 172.21.15.3:0/3570975716 shutdown_connections 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 --2- 172.21.15.3:0/3570975716 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c1075c0 0x7fa59c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 --2- 172.21.15.3:0/3570975716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c104c90 0x7fa59c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 --2- 172.21.15.3:0/3570975716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 0x7fa59c104750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 -- 172.21.15.3:0/3570975716 >> 172.21.15.3:0/3570975716 conn(0x7fa59c0fbea0 msgr2=0x7fa59c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 -- 172.21.15.3:0/3570975716 shutdown_connections 2024-08-01T04:16:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.951+0000 7fa5a3600640 1 -- 172.21.15.3:0/3570975716 wait complete. 2024-08-01T04:16:19.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.952+0000 7fa5a3600640 1 Processor -- start 2024-08-01T04:16:19.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 -- start start 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 0x7fa59c101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c104c90 0x7fa59c100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c1075c0 0x7fa59c100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10d520 con 0x7fa59c1075c0 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10d3a0 con 0x7fa59c104c90 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a3600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10d6a0 con 0x7fa59c102360 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c1075c0 0x7fa59c100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa59be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c104c90 0x7fa59c100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c1075c0 0x7fa59c100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53440/0 (socket says 172.21.15.3:53440) 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a0c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 0x7fa59c101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.953+0000 7fa5a1600640 1 -- 172.21.15.3:0/321225370 learned_addr learned my addr 172.21.15.3:0/321225370 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:19.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa59be00640 1 -- 172.21.15.3:0/321225370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 msgr2=0x7fa59c101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa59be00640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 0x7fa59c101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa59be00640 1 -- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c1075c0 msgr2=0x7fa59c100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa59be00640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c1075c0 0x7fa59c100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa59be00640 1 -- 172.21.15.3:0/321225370 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c1010e0 con 0x7fa59c104c90 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa59be00640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c104c90 0x7fa59c100310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa59000e9c0 tx=0x7fa59000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.954+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000cd80 con 0x7fa59c104c90 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.955+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5900035f0 con 0x7fa59c104c90 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.955+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c1a1c70 con 0x7fa59c104c90 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.955+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590010690 con 0x7fa59c104c90 2024-08-01T04:16:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.955+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1a21c0 con 0x7fa59c104c90 2024-08-01T04:16:19.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.956+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c0fd990 con 0x7fa59c104c90 2024-08-01T04:16:19.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.958+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7fa5900037a0 con 0x7fa59c104c90 2024-08-01T04:16:19.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.959+0000 7fa599600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa5840a2d00 0x7fa5840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:19.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.959+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa5900cf7e0 con 0x7fa59c104c90 2024-08-01T04:16:19.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.959+0000 7fa5a0c00640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa5840a2d00 0x7fa5840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:19.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.960+0000 7fa5a0c00640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa5840a2d00 0x7fa5840a51c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa58c031440 tx=0x7fa58c0315b0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:19.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:19.963+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa59009c360 con 0x7fa59c104c90 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: Unable to set osd_memory_target on smithi163 to 184780288: error parsing value: Value '184780288' is below minimum 939524096 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1389482119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3951613901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:16:20.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:19 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/716683090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-01T04:16:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-08-01T04:16:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: Unable to set osd_memory_target on smithi163 to 184780288: error parsing value: Value '184780288' is below minimum 939524096 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1389482119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3951613901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:16:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:19 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/716683090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-01T04:16:20.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.173+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa59c02d790 con 0x7fa59c104c90 2024-08-01T04:16:20.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.180+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7fa5900982d0 con 0x7fa59c104c90 2024-08-01T04:16:20.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.181+0000 7fa599600640 1 -- 172.21.15.3:0/321225370 <== mon.2 v2:172.21.15.160:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fa59005e800 con 0x7fa59c104c90 2024-08-01T04:16:20.183 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa5840a2d00 msgr2=0x7fa5840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa5840a2d00 0x7fa5840a51c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa58c031440 tx=0x7fa58c0315b0 comp rx=0 tx=0).stop 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c104c90 msgr2=0x7fa59c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c104c90 0x7fa59c100310 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa59000e9c0 tx=0x7fa59000ee80 comp rx=0 tx=0).stop 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 shutdown_connections 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa59c1075c0 0x7fa59c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:20.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa59c104c90 0x7fa59c100310 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa5840a2d00 0x7fa5840a51c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 --2- 172.21.15.3:0/321225370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa59c102360 0x7fa59c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 >> 172.21.15.3:0/321225370 conn(0x7fa59c0fbea0 msgr2=0x7fa59c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 shutdown_connections 2024-08-01T04:16:20.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:20.184+0000 7fa5a3600640 1 -- 172.21.15.3:0/321225370 wait complete. 2024-08-01T04:16:20.205 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-01T04:16:20.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: Unable to set osd_memory_target on smithi163 to 184780288: error parsing value: Value '184780288' is below minimum 939524096 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1389482119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:20.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3951613901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:16:20.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:19 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/716683090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-01T04:16:20.662 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:21.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 -- 172.21.15.3:0/2654451877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074c70 msgr2=0x7effb810dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:21.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 --2- 172.21.15.3:0/2654451877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074c70 0x7effb810dbb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effa4009900 tx=0x7effa402a480 comp rx=0 tx=0).stop 2024-08-01T04:16:21.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 -- 172.21.15.3:0/2654451877 shutdown_connections 2024-08-01T04:16:21.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 --2- 172.21.15.3:0/2654451877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb810e0f0 0x7effb8110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 --2- 172.21.15.3:0/2654451877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074c70 0x7effb810dbb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 --2- 172.21.15.3:0/2654451877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb8074350 0x7effb8074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 -- 172.21.15.3:0/2654451877 >> 172.21.15.3:0/2654451877 conn(0x7effb80fd5f0 msgr2=0x7effb80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:21.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.344+0000 7effbe000640 1 -- 172.21.15.3:0/2654451877 shutdown_connections 2024-08-01T04:16:21.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.345+0000 7effbe000640 1 -- 172.21.15.3:0/2654451877 wait complete. 2024-08-01T04:16:21.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 Processor -- start 2024-08-01T04:16:21.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 -- start start 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/321225370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi003 to 177.5M 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: Unable to set osd_memory_target on smithi003 to 186154393: error parsing value: Value '186154393' is below minimum 939524096 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:16:21.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:21 smithi003 ceph-mon[32262]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:16:21.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 0x7effb819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:21.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb8074c70 0x7effb819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:21.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb810e0f0 0x7effb819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effb8113eb0 con 0x7effb810e0f0 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7effb8113d30 con 0x7effb8074350 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.346+0000 7effbe000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effb8114030 con 0x7effb8074c70 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 0x7effb819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 0x7effb819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33432/0 (socket says 172.21.15.3:33432) 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 -- 172.21.15.3:0/511452818 learned_addr learned my addr 172.21.15.3:0/511452818 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7e00640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb810e0f0 0x7effb819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 -- 172.21.15.3:0/511452818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb8074c70 msgr2=0x7effb819a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb8074c70 0x7effb819a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 -- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb810e0f0 msgr2=0x7effb819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb810e0f0 0x7effb819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7400640 1 -- 172.21.15.3:0/511452818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb819f340 con 0x7effb8074350 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.347+0000 7effb7e00640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb810e0f0 0x7effb819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.348+0000 7effb7400640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 0x7effb819a390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7effa800e9f0 tx=0x7effa800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.348+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa800cd80 con 0x7effb8074350 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.348+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8010910 con 0x7effb8074350 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.348+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb819f630 con 0x7effb8074350 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.348+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8018950 con 0x7effb8074350 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.349+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb81a6f10 con 0x7effb8074350 2024-08-01T04:16:21.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.350+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb8074730 con 0x7effb8074350 2024-08-01T04:16:21.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.353+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7effa8018bd0 con 0x7effb8074350 2024-08-01T04:16:21.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.355+0000 7effafe00640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff9c0a2e90 0x7eff9c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.355+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7effa80cfbd0 con 0x7effb8074350 2024-08-01T04:16:21.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.355+0000 7effb6a00640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff9c0a2e90 0x7eff9c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:21.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.356+0000 7effb6a00640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff9c0a2e90 0x7eff9c0a5350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7effa402a9a0 tx=0x7effa4005fd0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:21.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.359+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7effa809c500 con 0x7effb8074350 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/321225370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi003 to 177.5M 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: Unable to set osd_memory_target on smithi003 to 186154393: error parsing value: Value '186154393' is below minimum 939524096 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:16:21.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:21 smithi163 ceph-mon[37383]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:16:21.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.633+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effb802d5e0 con 0x7effb8074350 2024-08-01T04:16:21.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.634+0000 7effafe00640 1 -- 172.21.15.3:0/511452818 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7effa809be00 con 0x7effb8074350 2024-08-01T04:16:21.636 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:21.636 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:16:21.636 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.636+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff9c0a2e90 msgr2=0x7eff9c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.636+0000 7effbe000640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff9c0a2e90 0x7eff9c0a5350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7effa402a9a0 tx=0x7effa4005fd0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.636+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 msgr2=0x7effb819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.636+0000 7effbe000640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 0x7effb819a390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7effa800e9f0 tx=0x7effa800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 shutdown_connections 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effb810e0f0 0x7effb819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff9c0a2e90 0x7eff9c0a5350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effb8074c70 0x7effb819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 --2- 172.21.15.3:0/511452818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effb8074350 0x7effb819a390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 >> 172.21.15.3:0/511452818 conn(0x7effb80fd5f0 msgr2=0x7effb80fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 shutdown_connections 2024-08-01T04:16:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:21.637+0000 7effbe000640 1 -- 172.21.15.3:0/511452818 wait complete. 2024-08-01T04:16:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-08-01T04:16:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/321225370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:16:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi003 to 177.5M 2024-08-01T04:16:21.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: Unable to set osd_memory_target on smithi003 to 186154393: error parsing value: Value '186154393' is below minimum 939524096 2024-08-01T04:16:21.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:21.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:21.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: Updating smithi003:/etc/ceph/ceph.conf 2024-08-01T04:16:21.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: Updating smithi160:/etc/ceph/ceph.conf 2024-08-01T04:16:21.654 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:21 smithi160 ceph-mon[37224]: Updating smithi163:/etc/ceph/ceph.conf 2024-08-01T04:16:22.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/511452818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.455 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-01T04:16:22.455 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-01T04:16:22.455 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:22.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/511452818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:22.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 -- 172.21.15.3:0/1580582938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 msgr2=0x7f3798102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 --2- 172.21.15.3:0/1580582938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f3798102ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3784009900 tx=0x7f378402a410 comp rx=0 tx=0).stop 2024-08-01T04:16:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 -- 172.21.15.3:0/1580582938 shutdown_connections 2024-08-01T04:16:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 --2- 172.21.15.3:0/1580582938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3798103410 0x7f3798109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 --2- 172.21.15.3:0/1580582938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f3798102ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 --2- 172.21.15.3:0/1580582938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 0x7f3798101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 -- 172.21.15.3:0/1580582938 >> 172.21.15.3:0/1580582938 conn(0x7f37980fcfe0 msgr2=0x7f37980ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 -- 172.21.15.3:0/1580582938 shutdown_connections 2024-08-01T04:16:22.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.637+0000 7f379fa00640 1 -- 172.21.15.3:0/1580582938 wait complete. 2024-08-01T04:16:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 Processor -- start 2024-08-01T04:16:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 -- start start 2024-08-01T04:16:22.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 0x7f3798199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f379819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3798103410 0x7f37981a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f379810d4e0 con 0x7f3798102a50 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f379810d360 con 0x7f3798103410 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.638+0000 7f379fa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f379810d660 con 0x7f3798101860 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f379819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f379d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 0x7f3798199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f379819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43572/0 (socket says 172.21.15.3:43572) 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f379d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 0x7f3798199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:36808/0 (socket says 172.21.15.3:36808) 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 -- 172.21.15.3:0/3987117491 learned_addr learned my addr 172.21.15.3:0/3987117491 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f379da00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3798103410 0x7f37981a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:22.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 -- 172.21.15.3:0/3987117491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 msgr2=0x7f3798199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:22.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 0x7f3798199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 -- 172.21.15.3:0/3987117491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3798103410 msgr2=0x7f37981a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:22.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3798103410 0x7f37981a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.639+0000 7f3797e00640 1 -- 172.21.15.3:0/3987117491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37981a1a20 con 0x7f3798102a50 2024-08-01T04:16:22.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.640+0000 7f3797e00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f379819a340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3784009230 tx=0x7f37840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.640+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378403f030 con 0x7f3798102a50 2024-08-01T04:16:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.640+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3784003ce0 con 0x7f3798102a50 2024-08-01T04:16:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.640+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37981a1cb0 con 0x7f3798102a50 2024-08-01T04:16:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.640+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784035070 con 0x7f3798102a50 2024-08-01T04:16:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.640+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37981a21a0 con 0x7f3798102a50 2024-08-01T04:16:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.641+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37981a2370 con 0x7f3798102a50 2024-08-01T04:16:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.643+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f3784042030 con 0x7f3798102a50 2024-08-01T04:16:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.645+0000 7f3795600640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f37800a2e20 0x7f37800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.645+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f37840efb40 con 0x7f3798102a50 2024-08-01T04:16:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.645+0000 7f379d000640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f37800a2e20 0x7f37800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.646+0000 7f379d000640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f37800a2e20 0x7f37800a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37981028b0 tx=0x7f378c005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.649+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f37840bbef0 con 0x7f3798102a50 2024-08-01T04:16:22.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.conf 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: Updating smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/511452818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:22.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.854+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3798109910 con 0x7f37800a2e20 2024-08-01T04:16:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.864+0000 7f3795600640 1 -- 172.21.15.3:0/3987117491 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258058 (secure 0 0 0) 0x7f3798109910 con 0x7f37800a2e20 2024-08-01T04:16:22.870 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:22.871 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f37800a2e20 msgr2=0x7f37800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f37800a2e20 0x7f37800a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37981028b0 tx=0x7f378c005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 msgr2=0x7f379819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f379819a340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3784009230 tx=0x7f37840042f0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 shutdown_connections 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3798103410 0x7f37981a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3798102a50 0x7f379819a340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f37800a2e20 0x7f37800a52e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 --2- 172.21.15.3:0/3987117491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3798101860 0x7f3798199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 >> 172.21.15.3:0/3987117491 conn(0x7f37980fcfe0 msgr2=0x7f37981081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 shutdown_connections 2024-08-01T04:16:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:22.869+0000 7f379fa00640 1 -- 172.21.15.3:0/3987117491 wait complete. 2024-08-01T04:16:22.890 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-08-01T04:16:21.864911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189523,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1027,"num_read_kb":675,"num_write":479,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365592,"kb_used_data":12232,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576232,"statfs":{"total":1151940427776,"available":1151566061568,"internally_reserved":0,"allocated":12525568,"data_stored":8996849,"data_compressed":329658,"data_compressed_allocated":442368,"data_compressed_original":3342336,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.983444"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551165+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:16:16.551165+0000","last_peered":"2024-08-01T04:16:16.551165+0000","last_clean":"2024-08-01T04:16:16.551165+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:16:16.551165+0000","last_undegraded":"2024-08-01T04:16:16.551165+0000","last_fullsized":"2024-08-01T04:16:16.551165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212873+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:16:16.212873+0000","last_peered":"2024-08-01T04:16:16.212873+0000","last_clean":"2024-08-01T04:16:16.212873+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:16:16.212873+0000","last_undegraded":"2024-08-01T04:16:16.212873+0000","last_fullsized":"2024-08-01T04:16:16.212873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212612+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:16:16.212612+0000","last_peered":"2024-08-01T04:16:16.212612+0000","last_clean":"2024-08-01T04:16:16.212612+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:16:16.212612+0000","last_undegraded":"2024-08-01T04:16:16.212612+0000","last_fullsized":"2024-08-01T04:16:16.212612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840852+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:16:15.840852+0000","last_peered":"2024-08-01T04:16:15.840852+0000","last_clean":"2024-08-01T04:16:15.840852+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:16:15.840852+0000","last_undegraded":"2024-08-01T04:16:15.840852+0000","last_fullsized":"2024-08-01T04:16:15.840852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953626+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:16:15.953626+0000","last_peered":"2024-08-01T04:16:15.953626+0000","last_clean":"2024-08-01T04:16:15.953626+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:16:15.953626+0000","last_undegraded":"2024-08-01T04:16:15.953626+0000","last_fullsized":"2024-08-01T04:16:15.953626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.966981+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:16:15.966981+0000","last_peered":"2024-08-01T04:16:15.966981+0000","last_clean":"2024-08-01T04:16:15.966981+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:16:15.966981+0000","last_undegraded":"2024-08-01T04:16:15.966981+0000","last_fullsized":"2024-08-01T04:16:15.966981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210071+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:16:16.210071+0000","last_peered":"2024-08-01T04:16:16.210071+0000","last_clean":"2024-08-01T04:16:16.210071+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:16:16.210071+0000","last_undegraded":"2024-08-01T04:16:16.210071+0000","last_fullsized":"2024-08-01T04:16:16.210071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879729+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:16:15.879729+0000","last_peered":"2024-08-01T04:16:15.879729+0000","last_clean":"2024-08-01T04:16:15.879729+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:16:15.879729+0000","last_undegraded":"2024-08-01T04:16:15.879729+0000","last_fullsized":"2024-08-01T04:16:15.879729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006303+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:16:16.006303+0000","last_peered":"2024-08-01T04:16:16.006303+0000","last_clean":"2024-08-01T04:16:16.006303+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:16:16.006303+0000","last_undegraded":"2024-08-01T04:16:16.006303+0000","last_fullsized":"2024-08-01T04:16:16.006303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967373+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:16:15.967373+0000","last_peered":"2024-08-01T04:16:15.967373+0000","last_clean":"2024-08-01T04:16:15.967373+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:16:15.967373+0000","last_undegraded":"2024-08-01T04:16:15.967373+0000","last_fullsized":"2024-08-01T04:16:15.967373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210488+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:16:16.210488+0000","last_peered":"2024-08-01T04:16:16.210488+0000","last_clean":"2024-08-01T04:16:16.210488+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:16:16.210488+0000","last_undegraded":"2024-08-01T04:16:16.210488+0000","last_fullsized":"2024-08-01T04:16:16.210488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841219+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:16:15.841219+0000","last_peered":"2024-08-01T04:16:15.841219+0000","last_clean":"2024-08-01T04:16:15.841219+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:16:15.841219+0000","last_undegraded":"2024-08-01T04:16:15.841219+0000","last_fullsized":"2024-08-01T04:16:15.841219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877759+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:16:15.877759+0000","last_peered":"2024-08-01T04:16:15.877759+0000","last_clean":"2024-08-01T04:16:15.877759+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:16:15.877759+0000","last_undegraded":"2024-08-01T04:16:15.877759+0000","last_fullsized":"2024-08-01T04:16:15.877759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211836+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:16:16.211836+0000","last_peered":"2024-08-01T04:16:16.211836+0000","last_clean":"2024-08-01T04:16:16.211836+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:16:16.211836+0000","last_undegraded":"2024-08-01T04:16:16.211836+0000","last_fullsized":"2024-08-01T04:16:16.211836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211595+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:16:16.211595+0000","last_peered":"2024-08-01T04:16:16.211595+0000","last_clean":"2024-08-01T04:16:16.211595+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:16:16.211595+0000","last_undegraded":"2024-08-01T04:16:16.211595+0000","last_fullsized":"2024-08-01T04:16:16.211595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005632+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:16:16.005632+0000","last_peered":"2024-08-01T04:16:16.005632+0000","last_clean":"2024-08-01T04:16:16.005632+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:16:16.005632+0000","last_undegraded":"2024-08-01T04:16:16.005632+0000","last_fullsized":"2024-08-01T04:16:16.005632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551271+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:16:16.551271+0000","last_peered":"2024-08-01T04:16:16.551271+0000","last_clean":"2024-08-01T04:16:16.551271+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:16:16.551271+0000","last_undegraded":"2024-08-01T04:16:16.551271+0000","last_fullsized":"2024-08-01T04:16:16.551271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840840+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:16:15.840840+0000","last_peered":"2024-08-01T04:16:15.840840+0000","last_clean":"2024-08-01T04:16:15.840840+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:16:15.840840+0000","last_undegraded":"2024-08-01T04:16:15.840840+0000","last_fullsized":"2024-08-01T04:16:15.840840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840916+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:16:15.840916+0000","last_peered":"2024-08-01T04:16:15.840916+0000","last_clean":"2024-08-01T04:16:15.840916+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:16:15.840916+0000","last_undegraded":"2024-08-01T04:16:15.840916+0000","last_fullsized":"2024-08-01T04:16:15.840916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550965+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:16:16.550965+0000","last_peered":"2024-08-01T04:16:16.550965+0000","last_clean":"2024-08-01T04:16:16.550965+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:16:16.550965+0000","last_undegraded":"2024-08-01T04:16:16.550965+0000","last_fullsized":"2024-08-01T04:16:16.550965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877859+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:16:15.877859+0000","last_peered":"2024-08-01T04:16:15.877859+0000","last_clean":"2024-08-01T04:16:15.877859+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:16:15.877859+0000","last_undegraded":"2024-08-01T04:16:15.877859+0000","last_fullsized":"2024-08-01T04:16:15.877859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006020+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:16:16.006020+0000","last_peered":"2024-08-01T04:16:16.006020+0000","last_clean":"2024-08-01T04:16:16.006020+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:16:16.006020+0000","last_undegraded":"2024-08-01T04:16:16.006020+0000","last_fullsized":"2024-08-01T04:16:16.006020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879660+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:16:15.879660+0000","last_peered":"2024-08-01T04:16:15.879660+0000","last_clean":"2024-08-01T04:16:15.879660+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:16:15.879660+0000","last_undegraded":"2024-08-01T04:16:15.879660+0000","last_fullsized":"2024-08-01T04:16:15.879660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953511+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:16:15.953511+0000","last_peered":"2024-08-01T04:16:15.953511+0000","last_clean":"2024-08-01T04:16:15.953511+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:16:15.953511+0000","last_undegraded":"2024-08-01T04:16:15.953511+0000","last_fullsized":"2024-08-01T04:16:15.953511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880212+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:16:15.880212+0000","last_peered":"2024-08-01T04:16:15.880212+0000","last_clean":"2024-08-01T04:16:15.880212+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:16:15.880212+0000","last_undegraded":"2024-08-01T04:16:15.880212+0000","last_fullsized":"2024-08-01T04:16:15.880212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840932+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:16:15.840932+0000","last_peered":"2024-08-01T04:16:15.840932+0000","last_clean":"2024-08-01T04:16:15.840932+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:16:15.840932+0000","last_undegraded":"2024-08-01T04:16:15.840932+0000","last_fullsized":"2024-08-01T04:16:15.840932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551039+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:16:16.551039+0000","last_peered":"2024-08-01T04:16:16.551039+0000","last_clean":"2024-08-01T04:16:16.551039+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:16:16.551039+0000","last_undegraded":"2024-08-01T04:16:16.551039+0000","last_fullsized":"2024-08-01T04:16:16.551039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877756+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:16:15.877756+0000","last_peered":"2024-08-01T04:16:15.877756+0000","last_clean":"2024-08-01T04:16:15.877756+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:16:15.877756+0000","last_undegraded":"2024-08-01T04:16:15.877756+0000","last_fullsized":"2024-08-01T04:16:15.877756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212112+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:16:16.212112+0000","last_peered":"2024-08-01T04:16:16.212112+0000","last_clean":"2024-08-01T04:16:16.212112+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:16:16.212112+0000","last_undegraded":"2024-08-01T04:16:16.212112+0000","last_fullsized":"2024-08-01T04:16:16.212112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840499+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:16:15.840499+0000","last_peered":"2024-08-01T04:16:15.840499+0000","last_clean":"2024-08-01T04:16:15.840499+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:16:15.840499+0000","last_undegraded":"2024-08-01T04:16:15.840499+0000","last_fullsized":"2024-08-01T04:16:15.840499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840592+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:16:15.840592+0000","last_peered":"2024-08-01T04:16:15.840592+0000","last_clean":"2024-08-01T04:16:15.840592+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:16:15.840592+0000","last_undegraded":"2024-08-01T04:16:15.840592+0000","last_fullsized":"2024-08-01T04:16:15.840592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550743+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:16:16.550743+0000","last_peered":"2024-08-01T04:16:16.550743+0000","last_clean":"2024-08-01T04:16:16.550743+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:16:16.550743+0000","last_undegraded":"2024-08-01T04:16:16.550743+0000","last_fullsized":"2024-08-01T04:16:16.550743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953539+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:16:15.953539+0000","last_peered":"2024-08-01T04:16:15.953539+0000","last_clean":"2024-08-01T04:16:15.953539+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:16:15.953539+0000","last_undegraded":"2024-08-01T04:16:15.953539+0000","last_fullsized":"2024-08-01T04:16:15.953539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":173,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.834559+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:16:16.834559+0000","last_peered":"2024-08-01T04:16:16.834559+0000","last_clean":"2024-08-01T04:16:16.834559+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:16:16.834559+0000","last_undegraded":"2024-08-01T04:16:16.834559+0000","last_fullsized":"2024-08-01T04:16:16.834559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963413+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:16:15.963413+0000","last_peered":"2024-08-01T04:16:15.963413+0000","last_clean":"2024-08-01T04:16:15.963413+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:16:15.963413+0000","last_undegraded":"2024-08-01T04:16:15.963413+0000","last_fullsized":"2024-08-01T04:16:15.963413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967038+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:16:15.967038+0000","last_peered":"2024-08-01T04:16:15.967038+0000","last_clean":"2024-08-01T04:16:15.967038+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:16:15.967038+0000","last_undegraded":"2024-08-01T04:16:15.967038+0000","last_fullsized":"2024-08-01T04:16:15.967038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877210+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:16:15.877210+0000","last_peered":"2024-08-01T04:16:15.877210+0000","last_clean":"2024-08-01T04:16:15.877210+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:16:15.877210+0000","last_undegraded":"2024-08-01T04:16:15.877210+0000","last_fullsized":"2024-08-01T04:16:15.877210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879844+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:16:15.879844+0000","last_peered":"2024-08-01T04:16:15.879844+0000","last_clean":"2024-08-01T04:16:15.879844+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:16:15.879844+0000","last_undegraded":"2024-08-01T04:16:15.879844+0000","last_fullsized":"2024-08-01T04:16:15.879844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963382+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:16:15.963382+0000","last_peered":"2024-08-01T04:16:15.963382+0000","last_clean":"2024-08-01T04:16:15.963382+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:16:15.963382+0000","last_undegraded":"2024-08-01T04:16:15.963382+0000","last_fullsized":"2024-08-01T04:16:15.963382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840510+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:16:15.840510+0000","last_peered":"2024-08-01T04:16:15.840510+0000","last_clean":"2024-08-01T04:16:15.840510+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:16:15.840510+0000","last_undegraded":"2024-08-01T04:16:15.840510+0000","last_fullsized":"2024-08-01T04:16:15.840510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.834039+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:16:16.834039+0000","last_peered":"2024-08-01T04:16:16.834039+0000","last_clean":"2024-08-01T04:16:16.834039+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:16:16.834039+0000","last_undegraded":"2024-08-01T04:16:16.834039+0000","last_fullsized":"2024-08-01T04:16:16.834039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006102+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:16:16.006102+0000","last_peered":"2024-08-01T04:16:16.006102+0000","last_clean":"2024-08-01T04:16:16.006102+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:16:16.006102+0000","last_undegraded":"2024-08-01T04:16:16.006102+0000","last_fullsized":"2024-08-01T04:16:16.006102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210517+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:16:16.210517+0000","last_peered":"2024-08-01T04:16:16.210517+0000","last_clean":"2024-08-01T04:16:16.210517+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:16:16.210517+0000","last_undegraded":"2024-08-01T04:16:16.210517+0000","last_fullsized":"2024-08-01T04:16:16.210517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213346+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:16:16.213346+0000","last_peered":"2024-08-01T04:16:16.213346+0000","last_clean":"2024-08-01T04:16:16.213346+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:16:16.213346+0000","last_undegraded":"2024-08-01T04:16:16.213346+0000","last_fullsized":"2024-08-01T04:16:16.213346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963763+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:16:15.963763+0000","last_peered":"2024-08-01T04:16:15.963763+0000","last_clean":"2024-08-01T04:16:15.963763+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:16:15.963763+0000","last_undegraded":"2024-08-01T04:16:15.963763+0000","last_fullsized":"2024-08-01T04:16:15.963763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005853+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:16:16.005853+0000","last_peered":"2024-08-01T04:16:16.005853+0000","last_clean":"2024-08-01T04:16:16.005853+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:16:16.005853+0000","last_undegraded":"2024-08-01T04:16:16.005853+0000","last_fullsized":"2024-08-01T04:16:16.005853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":333,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.835162+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:16:16.835162+0000","last_peered":"2024-08-01T04:16:16.835162+0000","last_clean":"2024-08-01T04:16:16.835162+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:16:16.835162+0000","last_undegraded":"2024-08-01T04:16:16.835162+0000","last_fullsized":"2024-08-01T04:16:16.835162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68653,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":266,"num_read_kb":199,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953989+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:16:15.953989+0000","last_peered":"2024-08-01T04:16:15.953989+0000","last_clean":"2024-08-01T04:16:15.953989+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:16:15.953989+0000","last_undegraded":"2024-08-01T04:16:15.953989+0000","last_fullsized":"2024-08-01T04:16:15.953989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840839+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:16:15.840839+0000","last_peered":"2024-08-01T04:16:15.840839+0000","last_clean":"2024-08-01T04:16:15.840839+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:16:15.840839+0000","last_undegraded":"2024-08-01T04:16:15.840839+0000","last_fullsized":"2024-08-01T04:16:15.840839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877540+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:16:15.877540+0000","last_peered":"2024-08-01T04:16:15.877540+0000","last_clean":"2024-08-01T04:16:15.877540+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:16:15.877540+0000","last_undegraded":"2024-08-01T04:16:15.877540+0000","last_fullsized":"2024-08-01T04:16:15.877540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963535+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:16:15.963535+0000","last_peered":"2024-08-01T04:16:15.963535+0000","last_clean":"2024-08-01T04:16:15.963535+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:16:15.963535+0000","last_undegraded":"2024-08-01T04:16:15.963535+0000","last_fullsized":"2024-08-01T04:16:15.963535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210232+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:16:16.210232+0000","last_peered":"2024-08-01T04:16:16.210232+0000","last_clean":"2024-08-01T04:16:16.210232+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:16:16.210232+0000","last_undegraded":"2024-08-01T04:16:16.210232+0000","last_fullsized":"2024-08-01T04:16:16.210232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.259331+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:16:16.259331+0000","last_peered":"2024-08-01T04:16:16.259331+0000","last_clean":"2024-08-01T04:16:16.259331+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:16:16.259331+0000","last_undegraded":"2024-08-01T04:16:16.259331+0000","last_fullsized":"2024-08-01T04:16:16.259331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210072+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:16:16.210072+0000","last_peered":"2024-08-01T04:16:16.210072+0000","last_clean":"2024-08-01T04:16:16.210072+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:16:16.210072+0000","last_undegraded":"2024-08-01T04:16:16.210072+0000","last_fullsized":"2024-08-01T04:16:16.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953840+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:16:15.953840+0000","last_peered":"2024-08-01T04:16:15.953840+0000","last_clean":"2024-08-01T04:16:15.953840+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:16:15.953840+0000","last_undegraded":"2024-08-01T04:16:15.953840+0000","last_fullsized":"2024-08-01T04:16:15.953840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963507+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:16:15.963507+0000","last_peered":"2024-08-01T04:16:15.963507+0000","last_clean":"2024-08-01T04:16:15.963507+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:16:15.963507+0000","last_undegraded":"2024-08-01T04:16:15.963507+0000","last_fullsized":"2024-08-01T04:16:15.963507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954076+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:16:15.954076+0000","last_peered":"2024-08-01T04:16:15.954076+0000","last_clean":"2024-08-01T04:16:15.954076+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:16:15.954076+0000","last_undegraded":"2024-08-01T04:16:15.954076+0000","last_fullsized":"2024-08-01T04:16:15.954076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551119+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:16:16.551119+0000","last_peered":"2024-08-01T04:16:16.551119+0000","last_clean":"2024-08-01T04:16:16.551119+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:16:16.551119+0000","last_undegraded":"2024-08-01T04:16:16.551119+0000","last_fullsized":"2024-08-01T04:16:16.551119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877775+0000","last_change":"2024-08-01T04:15:50.404577+0000","last_active":"2024-08-01T04:16:15.877775+0000","last_peered":"2024-08-01T04:16:15.877775+0000","last_clean":"2024-08-01T04:16:15.877775+0000","last_became_active":"2024-08-01T04:15:50.400018+0000","last_became_peered":"2024-08-01T04:15:50.400018+0000","last_unstale":"2024-08-01T04:16:15.877775+0000","last_undegraded":"2024-08-01T04:16:15.877775+0000","last_fullsized":"2024-08-01T04:16:15.877775+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:22:39.045462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210494+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:16:16.210494+0000","last_peered":"2024-08-01T04:16:16.210494+0000","last_clean":"2024-08-01T04:16:16.210494+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:16:16.210494+0000","last_undegraded":"2024-08-01T04:16:16.210494+0000","last_fullsized":"2024-08-01T04:16:16.210494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.163057+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:16:16.163057+0000","last_peered":"2024-08-01T04:16:16.163057+0000","last_clean":"2024-08-01T04:16:16.163057+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:16:16.163057+0000","last_undegraded":"2024-08-01T04:16:16.163057+0000","last_fullsized":"2024-08-01T04:16:16.163057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210202+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:16:16.210202+0000","last_peered":"2024-08-01T04:16:16.210202+0000","last_clean":"2024-08-01T04:16:16.210202+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:16:16.210202+0000","last_undegraded":"2024-08-01T04:16:16.210202+0000","last_fullsized":"2024-08-01T04:16:16.210202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841059+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:16:15.841059+0000","last_peered":"2024-08-01T04:16:15.841059+0000","last_clean":"2024-08-01T04:16:15.841059+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:16:15.841059+0000","last_undegraded":"2024-08-01T04:16:15.841059+0000","last_fullsized":"2024-08-01T04:16:15.841059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210445+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:16:16.210445+0000","last_peered":"2024-08-01T04:16:16.210445+0000","last_clean":"2024-08-01T04:16:16.210445+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:16:16.210445+0000","last_undegraded":"2024-08-01T04:16:16.210445+0000","last_fullsized":"2024-08-01T04:16:16.210445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210448+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:16:16.210448+0000","last_peered":"2024-08-01T04:16:16.210448+0000","last_clean":"2024-08-01T04:16:16.210448+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:16:16.210448+0000","last_undegraded":"2024-08-01T04:16:16.210448+0000","last_fullsized":"2024-08-01T04:16:16.210448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841282+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:16:15.841282+0000","last_peered":"2024-08-01T04:16:15.841282+0000","last_clean":"2024-08-01T04:16:15.841282+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:16:15.841282+0000","last_undegraded":"2024-08-01T04:16:15.841282+0000","last_fullsized":"2024-08-01T04:16:15.841282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880005+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:16:15.880005+0000","last_peered":"2024-08-01T04:16:15.880005+0000","last_clean":"2024-08-01T04:16:15.880005+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:16:15.880005+0000","last_undegraded":"2024-08-01T04:16:15.880005+0000","last_fullsized":"2024-08-01T04:16:15.880005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210288+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:16:16.210288+0000","last_peered":"2024-08-01T04:16:16.210288+0000","last_clean":"2024-08-01T04:16:16.210288+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:16:16.210288+0000","last_undegraded":"2024-08-01T04:16:16.210288+0000","last_fullsized":"2024-08-01T04:16:16.210288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877149+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:16:15.877149+0000","last_peered":"2024-08-01T04:16:15.877149+0000","last_clean":"2024-08-01T04:16:15.877149+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:16:15.877149+0000","last_undegraded":"2024-08-01T04:16:15.877149+0000","last_fullsized":"2024-08-01T04:16:15.877149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877440+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:16:15.877440+0000","last_peered":"2024-08-01T04:16:15.877440+0000","last_clean":"2024-08-01T04:16:15.877440+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:16:15.877440+0000","last_undegraded":"2024-08-01T04:16:15.877440+0000","last_fullsized":"2024-08-01T04:16:15.877440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963269+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:16:15.963269+0000","last_peered":"2024-08-01T04:16:15.963269+0000","last_clean":"2024-08-01T04:16:15.963269+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:16:15.963269+0000","last_undegraded":"2024-08-01T04:16:15.963269+0000","last_fullsized":"2024-08-01T04:16:15.963269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210461+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:16:16.210461+0000","last_peered":"2024-08-01T04:16:16.210461+0000","last_clean":"2024-08-01T04:16:16.210461+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:16:16.210461+0000","last_undegraded":"2024-08-01T04:16:16.210461+0000","last_fullsized":"2024-08-01T04:16:16.210461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967176+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:16:15.967176+0000","last_peered":"2024-08-01T04:16:15.967176+0000","last_clean":"2024-08-01T04:16:15.967176+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:16:15.967176+0000","last_undegraded":"2024-08-01T04:16:15.967176+0000","last_fullsized":"2024-08-01T04:16:15.967176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841074+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:16:15.841074+0000","last_peered":"2024-08-01T04:16:15.841074+0000","last_clean":"2024-08-01T04:16:15.841074+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:16:15.841074+0000","last_undegraded":"2024-08-01T04:16:15.841074+0000","last_fullsized":"2024-08-01T04:16:15.841074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967171+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:16:15.967171+0000","last_peered":"2024-08-01T04:16:15.967171+0000","last_clean":"2024-08-01T04:16:15.967171+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:16:15.967171+0000","last_undegraded":"2024-08-01T04:16:15.967171+0000","last_fullsized":"2024-08-01T04:16:15.967171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954025+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:16:15.954025+0000","last_peered":"2024-08-01T04:16:15.954025+0000","last_clean":"2024-08-01T04:16:15.954025+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:16:15.954025+0000","last_undegraded":"2024-08-01T04:16:15.954025+0000","last_fullsized":"2024-08-01T04:16:15.954025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877800+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:16:15.877800+0000","last_peered":"2024-08-01T04:16:15.877800+0000","last_clean":"2024-08-01T04:16:15.877800+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:16:15.877800+0000","last_undegraded":"2024-08-01T04:16:15.877800+0000","last_fullsized":"2024-08-01T04:16:15.877800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'48","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.172158+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:16:16.172158+0000","last_peered":"2024-08-01T04:16:16.172158+0000","last_clean":"2024-08-01T04:16:16.172158+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:16:16.172158+0000","last_undegraded":"2024-08-01T04:16:16.172158+0000","last_fullsized":"2024-08-01T04:16:16.172158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880558+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:16:15.880558+0000","last_peered":"2024-08-01T04:16:15.880558+0000","last_clean":"2024-08-01T04:16:15.880558+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:16:15.880558+0000","last_undegraded":"2024-08-01T04:16:15.880558+0000","last_fullsized":"2024-08-01T04:16:15.880558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841491+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:16:15.841491+0000","last_peered":"2024-08-01T04:16:15.841491+0000","last_clean":"2024-08-01T04:16:15.841491+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:16:15.841491+0000","last_undegraded":"2024-08-01T04:16:15.841491+0000","last_fullsized":"2024-08-01T04:16:15.841491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212996+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:16:16.212996+0000","last_peered":"2024-08-01T04:16:16.212996+0000","last_clean":"2024-08-01T04:16:16.212996+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:16:16.212996+0000","last_undegraded":"2024-08-01T04:16:16.212996+0000","last_fullsized":"2024-08-01T04:16:16.212996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005685+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:16:16.005685+0000","last_peered":"2024-08-01T04:16:16.005685+0000","last_clean":"2024-08-01T04:16:16.005685+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:16:16.005685+0000","last_undegraded":"2024-08-01T04:16:16.005685+0000","last_fullsized":"2024-08-01T04:16:16.005685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953515+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:16:15.953515+0000","last_peered":"2024-08-01T04:16:15.953515+0000","last_clean":"2024-08-01T04:16:15.953515+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:16:15.953515+0000","last_undegraded":"2024-08-01T04:16:15.953515+0000","last_fullsized":"2024-08-01T04:16:15.953515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879984+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:16:15.879984+0000","last_peered":"2024-08-01T04:16:15.879984+0000","last_clean":"2024-08-01T04:16:15.879984+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:16:15.879984+0000","last_undegraded":"2024-08-01T04:16:15.879984+0000","last_fullsized":"2024-08-01T04:16:15.879984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550822+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:16:16.550822+0000","last_peered":"2024-08-01T04:16:16.550822+0000","last_clean":"2024-08-01T04:16:16.550822+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:16:16.550822+0000","last_undegraded":"2024-08-01T04:16:16.550822+0000","last_fullsized":"2024-08-01T04:16:16.550822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550847+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:16:16.550847+0000","last_peered":"2024-08-01T04:16:16.550847+0000","last_clean":"2024-08-01T04:16:16.550847+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:16:16.550847+0000","last_undegraded":"2024-08-01T04:16:16.550847+0000","last_fullsized":"2024-08-01T04:16:16.550847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880233+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:16:15.880233+0000","last_peered":"2024-08-01T04:16:15.880233+0000","last_clean":"2024-08-01T04:16:15.880233+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:16:15.880233+0000","last_undegraded":"2024-08-01T04:16:15.880233+0000","last_fullsized":"2024-08-01T04:16:15.880233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953606+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:16:15.953606+0000","last_peered":"2024-08-01T04:16:15.953606+0000","last_clean":"2024-08-01T04:16:15.953606+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:16:15.953606+0000","last_undegraded":"2024-08-01T04:16:15.953606+0000","last_fullsized":"2024-08-01T04:16:15.953606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210241+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:16:16.210241+0000","last_peered":"2024-08-01T04:16:16.210241+0000","last_clean":"2024-08-01T04:16:16.210241+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:16:16.210241+0000","last_undegraded":"2024-08-01T04:16:16.210241+0000","last_fullsized":"2024-08-01T04:16:16.210241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963633+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:16:15.963633+0000","last_peered":"2024-08-01T04:16:15.963633+0000","last_clean":"2024-08-01T04:16:15.963633+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:16:15.963633+0000","last_undegraded":"2024-08-01T04:16:15.963633+0000","last_fullsized":"2024-08-01T04:16:15.963633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953850+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:16:15.953850+0000","last_peered":"2024-08-01T04:16:15.953850+0000","last_clean":"2024-08-01T04:16:15.953850+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:16:15.953850+0000","last_undegraded":"2024-08-01T04:16:15.953850+0000","last_fullsized":"2024-08-01T04:16:15.953850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005670+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:16:16.005670+0000","last_peered":"2024-08-01T04:16:16.005670+0000","last_clean":"2024-08-01T04:16:16.005670+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:16:16.005670+0000","last_undegraded":"2024-08-01T04:16:16.005670+0000","last_fullsized":"2024-08-01T04:16:16.005670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550806+0000","last_change":"2024-08-01T04:15:50.403853+0000","last_active":"2024-08-01T04:16:16.550806+0000","last_peered":"2024-08-01T04:16:16.550806+0000","last_clean":"2024-08-01T04:16:16.550806+0000","last_became_active":"2024-08-01T04:15:50.400282+0000","last_became_peered":"2024-08-01T04:15:50.400282+0000","last_unstale":"2024-08-01T04:16:16.550806+0000","last_undegraded":"2024-08-01T04:16:16.550806+0000","last_fullsized":"2024-08-01T04:16:16.550806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:11:52.649176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213209+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:16:16.213209+0000","last_peered":"2024-08-01T04:16:16.213209+0000","last_clean":"2024-08-01T04:16:16.213209+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:16:16.213209+0000","last_undegraded":"2024-08-01T04:16:16.213209+0000","last_fullsized":"2024-08-01T04:16:16.213209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880724+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:16:15.880724+0000","last_peered":"2024-08-01T04:16:15.880724+0000","last_clean":"2024-08-01T04:16:15.880724+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:16:15.880724+0000","last_undegraded":"2024-08-01T04:16:15.880724+0000","last_fullsized":"2024-08-01T04:16:15.880724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880592+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:16:15.880592+0000","last_peered":"2024-08-01T04:16:15.880592+0000","last_clean":"2024-08-01T04:16:15.880592+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:16:15.880592+0000","last_undegraded":"2024-08-01T04:16:15.880592+0000","last_fullsized":"2024-08-01T04:16:15.880592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840998+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:16:15.840998+0000","last_peered":"2024-08-01T04:16:15.840998+0000","last_clean":"2024-08-01T04:16:15.840998+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:16:15.840998+0000","last_undegraded":"2024-08-01T04:16:15.840998+0000","last_fullsized":"2024-08-01T04:16:15.840998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598921,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":891,"num_read_kb":556,"num_write":296,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1796763,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449129,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":484724,"data_compressed":11581,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678063,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1167360,"data_stored":1280269,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":887219,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103226,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":887139,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1077481,"data_compressed":47455,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":493392,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":1217667,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":494750,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":559751,"data_compressed":17833,"data_compressed_allocated":24576,"data_compressed_original":106496,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":557956,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":561838,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":494663,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133997,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202752,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:16:22.895 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.118+0000 7f7d78000640 1 -- 172.21.15.3:0/3916411838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d70109a00 msgr2=0x7f7d70109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.118+0000 7f7d78000640 1 --2- 172.21.15.3:0/3916411838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d70109a00 0x7f7d70109de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009860 tx=0x7f7d6402a340 comp rx=0 tx=0).stop 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 -- 172.21.15.3:0/3916411838 shutdown_connections 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 --2- 172.21.15.3:0/3916411838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d70104390 0x7f7d701108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 --2- 172.21.15.3:0/3916411838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d701039f0 0x7f7d70103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 --2- 172.21.15.3:0/3916411838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d70109a00 0x7f7d70109de0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 -- 172.21.15.3:0/3916411838 >> 172.21.15.3:0/3916411838 conn(0x7f7d700ff6d0 msgr2=0x7f7d70101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 -- 172.21.15.3:0/3916411838 shutdown_connections 2024-08-01T04:16:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.119+0000 7f7d78000640 1 -- 172.21.15.3:0/3916411838 wait complete. 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 Processor -- start 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 -- start start 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 0x7f7d7019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d70104390 0x7f7d7019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d70109a00 0x7f7d7019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70113e40 con 0x7f7d70104390 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70113cc0 con 0x7f7d701039f0 2024-08-01T04:16:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d78000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70113fc0 con 0x7f7d70109a00 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.120+0000 7f7d74c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d70104390 0x7f7d7019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 0x7f7d7019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 0x7f7d7019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33726/0 (socket says 172.21.15.3:33726) 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 -- 172.21.15.3:0/2982368893 learned_addr learned my addr 172.21.15.3:0/2982368893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d76000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d70109a00 0x7f7d7019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 -- 172.21.15.3:0/2982368893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d70109a00 msgr2=0x7f7d7019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d70109a00 0x7f7d7019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 -- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d70104390 msgr2=0x7f7d7019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d70104390 0x7f7d7019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d75600640 1 -- 172.21.15.3:0/2982368893 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7019f360 con 0x7f7d701039f0 2024-08-01T04:16:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.121+0000 7f7d76000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d70109a00 0x7f7d7019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.122+0000 7f7d75600640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 0x7f7d7019a370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7d64035040 tx=0x7f7d64033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.122+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64037040 con 0x7f7d701039f0 2024-08-01T04:16:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.122+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d64040070 con 0x7f7d701039f0 2024-08-01T04:16:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.122+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64003a10 con 0x7f7d701039f0 2024-08-01T04:16:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.122+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7019f5f0 con 0x7f7d701039f0 2024-08-01T04:16:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.122+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701a6ee0 con 0x7f7d701039f0 2024-08-01T04:16:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.123+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d70103e50 con 0x7f7d701039f0 2024-08-01T04:16:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.125+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f7d64004150 con 0x7f7d701039f0 2024-08-01T04:16:23.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.127+0000 7f7d6e000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d580a2ea0 0x7f7d580a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.127+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f7d640ef5b0 con 0x7f7d701039f0 2024-08-01T04:16:23.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.127+0000 7f7d74c00640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d580a2ea0 0x7f7d580a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.128+0000 7f7d74c00640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d580a2ea0 0x7f7d580a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d70101cc0 tx=0x7f7d5c005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:23.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.131+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7d640bbff0 con 0x7f7d701039f0 2024-08-01T04:16:23.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.339+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d70062360 con 0x7f7d580a2ea0 2024-08-01T04:16:23.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.347+0000 7f7d6e000640 1 -- 172.21.15.3:0/2982368893 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258058 (secure 0 0 0) 0x7f7d70062360 con 0x7f7d580a2ea0 2024-08-01T04:16:23.352 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:23.353 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d580a2ea0 msgr2=0x7f7d580a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d580a2ea0 0x7f7d580a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d70101cc0 tx=0x7f7d5c005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 msgr2=0x7f7d7019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 0x7f7d7019a370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7d64035040 tx=0x7f7d64033b60 comp rx=0 tx=0).stop 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 shutdown_connections 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d580a2ea0 0x7f7d580a5360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d70109a00 0x7f7d7019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d70104390 0x7f7d7019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 --2- 172.21.15.3:0/2982368893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d701039f0 0x7f7d7019a370 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 >> 172.21.15.3:0/2982368893 conn(0x7f7d700ff6d0 msgr2=0x7f7d700fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:23.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.352+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 shutdown_connections 2024-08-01T04:16:23.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.353+0000 7f7d78000640 1 -- 172.21.15.3:0/2982368893 wait complete. 2024-08-01T04:16:23.375 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-08-01T04:16:21.864911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189523,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1027,"num_read_kb":675,"num_write":479,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365592,"kb_used_data":12232,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576232,"statfs":{"total":1151940427776,"available":1151566061568,"internally_reserved":0,"allocated":12525568,"data_stored":8996849,"data_compressed":329658,"data_compressed_allocated":442368,"data_compressed_original":3342336,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.983444"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551165+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:16:16.551165+0000","last_peered":"2024-08-01T04:16:16.551165+0000","last_clean":"2024-08-01T04:16:16.551165+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:16:16.551165+0000","last_undegraded":"2024-08-01T04:16:16.551165+0000","last_fullsized":"2024-08-01T04:16:16.551165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212873+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:16:16.212873+0000","last_peered":"2024-08-01T04:16:16.212873+0000","last_clean":"2024-08-01T04:16:16.212873+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:16:16.212873+0000","last_undegraded":"2024-08-01T04:16:16.212873+0000","last_fullsized":"2024-08-01T04:16:16.212873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212612+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:16:16.212612+0000","last_peered":"2024-08-01T04:16:16.212612+0000","last_clean":"2024-08-01T04:16:16.212612+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:16:16.212612+0000","last_undegraded":"2024-08-01T04:16:16.212612+0000","last_fullsized":"2024-08-01T04:16:16.212612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840852+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:16:15.840852+0000","last_peered":"2024-08-01T04:16:15.840852+0000","last_clean":"2024-08-01T04:16:15.840852+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:16:15.840852+0000","last_undegraded":"2024-08-01T04:16:15.840852+0000","last_fullsized":"2024-08-01T04:16:15.840852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953626+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:16:15.953626+0000","last_peered":"2024-08-01T04:16:15.953626+0000","last_clean":"2024-08-01T04:16:15.953626+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:16:15.953626+0000","last_undegraded":"2024-08-01T04:16:15.953626+0000","last_fullsized":"2024-08-01T04:16:15.953626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.966981+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:16:15.966981+0000","last_peered":"2024-08-01T04:16:15.966981+0000","last_clean":"2024-08-01T04:16:15.966981+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:16:15.966981+0000","last_undegraded":"2024-08-01T04:16:15.966981+0000","last_fullsized":"2024-08-01T04:16:15.966981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210071+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:16:16.210071+0000","last_peered":"2024-08-01T04:16:16.210071+0000","last_clean":"2024-08-01T04:16:16.210071+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:16:16.210071+0000","last_undegraded":"2024-08-01T04:16:16.210071+0000","last_fullsized":"2024-08-01T04:16:16.210071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879729+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:16:15.879729+0000","last_peered":"2024-08-01T04:16:15.879729+0000","last_clean":"2024-08-01T04:16:15.879729+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:16:15.879729+0000","last_undegraded":"2024-08-01T04:16:15.879729+0000","last_fullsized":"2024-08-01T04:16:15.879729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006303+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:16:16.006303+0000","last_peered":"2024-08-01T04:16:16.006303+0000","last_clean":"2024-08-01T04:16:16.006303+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:16:16.006303+0000","last_undegraded":"2024-08-01T04:16:16.006303+0000","last_fullsized":"2024-08-01T04:16:16.006303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967373+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:16:15.967373+0000","last_peered":"2024-08-01T04:16:15.967373+0000","last_clean":"2024-08-01T04:16:15.967373+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:16:15.967373+0000","last_undegraded":"2024-08-01T04:16:15.967373+0000","last_fullsized":"2024-08-01T04:16:15.967373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210488+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:16:16.210488+0000","last_peered":"2024-08-01T04:16:16.210488+0000","last_clean":"2024-08-01T04:16:16.210488+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:16:16.210488+0000","last_undegraded":"2024-08-01T04:16:16.210488+0000","last_fullsized":"2024-08-01T04:16:16.210488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841219+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:16:15.841219+0000","last_peered":"2024-08-01T04:16:15.841219+0000","last_clean":"2024-08-01T04:16:15.841219+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:16:15.841219+0000","last_undegraded":"2024-08-01T04:16:15.841219+0000","last_fullsized":"2024-08-01T04:16:15.841219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877759+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:16:15.877759+0000","last_peered":"2024-08-01T04:16:15.877759+0000","last_clean":"2024-08-01T04:16:15.877759+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:16:15.877759+0000","last_undegraded":"2024-08-01T04:16:15.877759+0000","last_fullsized":"2024-08-01T04:16:15.877759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211836+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:16:16.211836+0000","last_peered":"2024-08-01T04:16:16.211836+0000","last_clean":"2024-08-01T04:16:16.211836+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:16:16.211836+0000","last_undegraded":"2024-08-01T04:16:16.211836+0000","last_fullsized":"2024-08-01T04:16:16.211836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211595+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:16:16.211595+0000","last_peered":"2024-08-01T04:16:16.211595+0000","last_clean":"2024-08-01T04:16:16.211595+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:16:16.211595+0000","last_undegraded":"2024-08-01T04:16:16.211595+0000","last_fullsized":"2024-08-01T04:16:16.211595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005632+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:16:16.005632+0000","last_peered":"2024-08-01T04:16:16.005632+0000","last_clean":"2024-08-01T04:16:16.005632+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:16:16.005632+0000","last_undegraded":"2024-08-01T04:16:16.005632+0000","last_fullsized":"2024-08-01T04:16:16.005632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551271+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:16:16.551271+0000","last_peered":"2024-08-01T04:16:16.551271+0000","last_clean":"2024-08-01T04:16:16.551271+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:16:16.551271+0000","last_undegraded":"2024-08-01T04:16:16.551271+0000","last_fullsized":"2024-08-01T04:16:16.551271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840840+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:16:15.840840+0000","last_peered":"2024-08-01T04:16:15.840840+0000","last_clean":"2024-08-01T04:16:15.840840+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:16:15.840840+0000","last_undegraded":"2024-08-01T04:16:15.840840+0000","last_fullsized":"2024-08-01T04:16:15.840840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840916+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:16:15.840916+0000","last_peered":"2024-08-01T04:16:15.840916+0000","last_clean":"2024-08-01T04:16:15.840916+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:16:15.840916+0000","last_undegraded":"2024-08-01T04:16:15.840916+0000","last_fullsized":"2024-08-01T04:16:15.840916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550965+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:16:16.550965+0000","last_peered":"2024-08-01T04:16:16.550965+0000","last_clean":"2024-08-01T04:16:16.550965+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:16:16.550965+0000","last_undegraded":"2024-08-01T04:16:16.550965+0000","last_fullsized":"2024-08-01T04:16:16.550965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877859+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:16:15.877859+0000","last_peered":"2024-08-01T04:16:15.877859+0000","last_clean":"2024-08-01T04:16:15.877859+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:16:15.877859+0000","last_undegraded":"2024-08-01T04:16:15.877859+0000","last_fullsized":"2024-08-01T04:16:15.877859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006020+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:16:16.006020+0000","last_peered":"2024-08-01T04:16:16.006020+0000","last_clean":"2024-08-01T04:16:16.006020+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:16:16.006020+0000","last_undegraded":"2024-08-01T04:16:16.006020+0000","last_fullsized":"2024-08-01T04:16:16.006020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879660+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:16:15.879660+0000","last_peered":"2024-08-01T04:16:15.879660+0000","last_clean":"2024-08-01T04:16:15.879660+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:16:15.879660+0000","last_undegraded":"2024-08-01T04:16:15.879660+0000","last_fullsized":"2024-08-01T04:16:15.879660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953511+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:16:15.953511+0000","last_peered":"2024-08-01T04:16:15.953511+0000","last_clean":"2024-08-01T04:16:15.953511+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:16:15.953511+0000","last_undegraded":"2024-08-01T04:16:15.953511+0000","last_fullsized":"2024-08-01T04:16:15.953511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880212+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:16:15.880212+0000","last_peered":"2024-08-01T04:16:15.880212+0000","last_clean":"2024-08-01T04:16:15.880212+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:16:15.880212+0000","last_undegraded":"2024-08-01T04:16:15.880212+0000","last_fullsized":"2024-08-01T04:16:15.880212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840932+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:16:15.840932+0000","last_peered":"2024-08-01T04:16:15.840932+0000","last_clean":"2024-08-01T04:16:15.840932+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:16:15.840932+0000","last_undegraded":"2024-08-01T04:16:15.840932+0000","last_fullsized":"2024-08-01T04:16:15.840932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551039+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:16:16.551039+0000","last_peered":"2024-08-01T04:16:16.551039+0000","last_clean":"2024-08-01T04:16:16.551039+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:16:16.551039+0000","last_undegraded":"2024-08-01T04:16:16.551039+0000","last_fullsized":"2024-08-01T04:16:16.551039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877756+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:16:15.877756+0000","last_peered":"2024-08-01T04:16:15.877756+0000","last_clean":"2024-08-01T04:16:15.877756+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:16:15.877756+0000","last_undegraded":"2024-08-01T04:16:15.877756+0000","last_fullsized":"2024-08-01T04:16:15.877756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212112+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:16:16.212112+0000","last_peered":"2024-08-01T04:16:16.212112+0000","last_clean":"2024-08-01T04:16:16.212112+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:16:16.212112+0000","last_undegraded":"2024-08-01T04:16:16.212112+0000","last_fullsized":"2024-08-01T04:16:16.212112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840499+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:16:15.840499+0000","last_peered":"2024-08-01T04:16:15.840499+0000","last_clean":"2024-08-01T04:16:15.840499+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:16:15.840499+0000","last_undegraded":"2024-08-01T04:16:15.840499+0000","last_fullsized":"2024-08-01T04:16:15.840499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840592+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:16:15.840592+0000","last_peered":"2024-08-01T04:16:15.840592+0000","last_clean":"2024-08-01T04:16:15.840592+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:16:15.840592+0000","last_undegraded":"2024-08-01T04:16:15.840592+0000","last_fullsized":"2024-08-01T04:16:15.840592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550743+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:16:16.550743+0000","last_peered":"2024-08-01T04:16:16.550743+0000","last_clean":"2024-08-01T04:16:16.550743+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:16:16.550743+0000","last_undegraded":"2024-08-01T04:16:16.550743+0000","last_fullsized":"2024-08-01T04:16:16.550743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953539+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:16:15.953539+0000","last_peered":"2024-08-01T04:16:15.953539+0000","last_clean":"2024-08-01T04:16:15.953539+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:16:15.953539+0000","last_undegraded":"2024-08-01T04:16:15.953539+0000","last_fullsized":"2024-08-01T04:16:15.953539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":173,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.834559+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:16:16.834559+0000","last_peered":"2024-08-01T04:16:16.834559+0000","last_clean":"2024-08-01T04:16:16.834559+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:16:16.834559+0000","last_undegraded":"2024-08-01T04:16:16.834559+0000","last_fullsized":"2024-08-01T04:16:16.834559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963413+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:16:15.963413+0000","last_peered":"2024-08-01T04:16:15.963413+0000","last_clean":"2024-08-01T04:16:15.963413+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:16:15.963413+0000","last_undegraded":"2024-08-01T04:16:15.963413+0000","last_fullsized":"2024-08-01T04:16:15.963413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967038+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:16:15.967038+0000","last_peered":"2024-08-01T04:16:15.967038+0000","last_clean":"2024-08-01T04:16:15.967038+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:16:15.967038+0000","last_undegraded":"2024-08-01T04:16:15.967038+0000","last_fullsized":"2024-08-01T04:16:15.967038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877210+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:16:15.877210+0000","last_peered":"2024-08-01T04:16:15.877210+0000","last_clean":"2024-08-01T04:16:15.877210+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:16:15.877210+0000","last_undegraded":"2024-08-01T04:16:15.877210+0000","last_fullsized":"2024-08-01T04:16:15.877210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879844+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:16:15.879844+0000","last_peered":"2024-08-01T04:16:15.879844+0000","last_clean":"2024-08-01T04:16:15.879844+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:16:15.879844+0000","last_undegraded":"2024-08-01T04:16:15.879844+0000","last_fullsized":"2024-08-01T04:16:15.879844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963382+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:16:15.963382+0000","last_peered":"2024-08-01T04:16:15.963382+0000","last_clean":"2024-08-01T04:16:15.963382+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:16:15.963382+0000","last_undegraded":"2024-08-01T04:16:15.963382+0000","last_fullsized":"2024-08-01T04:16:15.963382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840510+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:16:15.840510+0000","last_peered":"2024-08-01T04:16:15.840510+0000","last_clean":"2024-08-01T04:16:15.840510+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:16:15.840510+0000","last_undegraded":"2024-08-01T04:16:15.840510+0000","last_fullsized":"2024-08-01T04:16:15.840510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.834039+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:16:16.834039+0000","last_peered":"2024-08-01T04:16:16.834039+0000","last_clean":"2024-08-01T04:16:16.834039+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:16:16.834039+0000","last_undegraded":"2024-08-01T04:16:16.834039+0000","last_fullsized":"2024-08-01T04:16:16.834039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006102+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:16:16.006102+0000","last_peered":"2024-08-01T04:16:16.006102+0000","last_clean":"2024-08-01T04:16:16.006102+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:16:16.006102+0000","last_undegraded":"2024-08-01T04:16:16.006102+0000","last_fullsized":"2024-08-01T04:16:16.006102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210517+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:16:16.210517+0000","last_peered":"2024-08-01T04:16:16.210517+0000","last_clean":"2024-08-01T04:16:16.210517+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:16:16.210517+0000","last_undegraded":"2024-08-01T04:16:16.210517+0000","last_fullsized":"2024-08-01T04:16:16.210517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213346+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:16:16.213346+0000","last_peered":"2024-08-01T04:16:16.213346+0000","last_clean":"2024-08-01T04:16:16.213346+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:16:16.213346+0000","last_undegraded":"2024-08-01T04:16:16.213346+0000","last_fullsized":"2024-08-01T04:16:16.213346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963763+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:16:15.963763+0000","last_peered":"2024-08-01T04:16:15.963763+0000","last_clean":"2024-08-01T04:16:15.963763+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:16:15.963763+0000","last_undegraded":"2024-08-01T04:16:15.963763+0000","last_fullsized":"2024-08-01T04:16:15.963763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005853+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:16:16.005853+0000","last_peered":"2024-08-01T04:16:16.005853+0000","last_clean":"2024-08-01T04:16:16.005853+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:16:16.005853+0000","last_undegraded":"2024-08-01T04:16:16.005853+0000","last_fullsized":"2024-08-01T04:16:16.005853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":333,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.835162+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:16:16.835162+0000","last_peered":"2024-08-01T04:16:16.835162+0000","last_clean":"2024-08-01T04:16:16.835162+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:16:16.835162+0000","last_undegraded":"2024-08-01T04:16:16.835162+0000","last_fullsized":"2024-08-01T04:16:16.835162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68653,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":266,"num_read_kb":199,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953989+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:16:15.953989+0000","last_peered":"2024-08-01T04:16:15.953989+0000","last_clean":"2024-08-01T04:16:15.953989+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:16:15.953989+0000","last_undegraded":"2024-08-01T04:16:15.953989+0000","last_fullsized":"2024-08-01T04:16:15.953989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840839+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:16:15.840839+0000","last_peered":"2024-08-01T04:16:15.840839+0000","last_clean":"2024-08-01T04:16:15.840839+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:16:15.840839+0000","last_undegraded":"2024-08-01T04:16:15.840839+0000","last_fullsized":"2024-08-01T04:16:15.840839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877540+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:16:15.877540+0000","last_peered":"2024-08-01T04:16:15.877540+0000","last_clean":"2024-08-01T04:16:15.877540+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:16:15.877540+0000","last_undegraded":"2024-08-01T04:16:15.877540+0000","last_fullsized":"2024-08-01T04:16:15.877540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963535+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:16:15.963535+0000","last_peered":"2024-08-01T04:16:15.963535+0000","last_clean":"2024-08-01T04:16:15.963535+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:16:15.963535+0000","last_undegraded":"2024-08-01T04:16:15.963535+0000","last_fullsized":"2024-08-01T04:16:15.963535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210232+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:16:16.210232+0000","last_peered":"2024-08-01T04:16:16.210232+0000","last_clean":"2024-08-01T04:16:16.210232+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:16:16.210232+0000","last_undegraded":"2024-08-01T04:16:16.210232+0000","last_fullsized":"2024-08-01T04:16:16.210232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.259331+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:16:16.259331+0000","last_peered":"2024-08-01T04:16:16.259331+0000","last_clean":"2024-08-01T04:16:16.259331+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:16:16.259331+0000","last_undegraded":"2024-08-01T04:16:16.259331+0000","last_fullsized":"2024-08-01T04:16:16.259331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210072+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:16:16.210072+0000","last_peered":"2024-08-01T04:16:16.210072+0000","last_clean":"2024-08-01T04:16:16.210072+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:16:16.210072+0000","last_undegraded":"2024-08-01T04:16:16.210072+0000","last_fullsized":"2024-08-01T04:16:16.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953840+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:16:15.953840+0000","last_peered":"2024-08-01T04:16:15.953840+0000","last_clean":"2024-08-01T04:16:15.953840+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:16:15.953840+0000","last_undegraded":"2024-08-01T04:16:15.953840+0000","last_fullsized":"2024-08-01T04:16:15.953840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963507+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:16:15.963507+0000","last_peered":"2024-08-01T04:16:15.963507+0000","last_clean":"2024-08-01T04:16:15.963507+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:16:15.963507+0000","last_undegraded":"2024-08-01T04:16:15.963507+0000","last_fullsized":"2024-08-01T04:16:15.963507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954076+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:16:15.954076+0000","last_peered":"2024-08-01T04:16:15.954076+0000","last_clean":"2024-08-01T04:16:15.954076+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:16:15.954076+0000","last_undegraded":"2024-08-01T04:16:15.954076+0000","last_fullsized":"2024-08-01T04:16:15.954076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551119+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:16:16.551119+0000","last_peered":"2024-08-01T04:16:16.551119+0000","last_clean":"2024-08-01T04:16:16.551119+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:16:16.551119+0000","last_undegraded":"2024-08-01T04:16:16.551119+0000","last_fullsized":"2024-08-01T04:16:16.551119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877775+0000","last_change":"2024-08-01T04:15:50.404577+0000","last_active":"2024-08-01T04:16:15.877775+0000","last_peered":"2024-08-01T04:16:15.877775+0000","last_clean":"2024-08-01T04:16:15.877775+0000","last_became_active":"2024-08-01T04:15:50.400018+0000","last_became_peered":"2024-08-01T04:15:50.400018+0000","last_unstale":"2024-08-01T04:16:15.877775+0000","last_undegraded":"2024-08-01T04:16:15.877775+0000","last_fullsized":"2024-08-01T04:16:15.877775+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:22:39.045462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210494+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:16:16.210494+0000","last_peered":"2024-08-01T04:16:16.210494+0000","last_clean":"2024-08-01T04:16:16.210494+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:16:16.210494+0000","last_undegraded":"2024-08-01T04:16:16.210494+0000","last_fullsized":"2024-08-01T04:16:16.210494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.163057+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:16:16.163057+0000","last_peered":"2024-08-01T04:16:16.163057+0000","last_clean":"2024-08-01T04:16:16.163057+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:16:16.163057+0000","last_undegraded":"2024-08-01T04:16:16.163057+0000","last_fullsized":"2024-08-01T04:16:16.163057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210202+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:16:16.210202+0000","last_peered":"2024-08-01T04:16:16.210202+0000","last_clean":"2024-08-01T04:16:16.210202+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:16:16.210202+0000","last_undegraded":"2024-08-01T04:16:16.210202+0000","last_fullsized":"2024-08-01T04:16:16.210202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841059+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:16:15.841059+0000","last_peered":"2024-08-01T04:16:15.841059+0000","last_clean":"2024-08-01T04:16:15.841059+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:16:15.841059+0000","last_undegraded":"2024-08-01T04:16:15.841059+0000","last_fullsized":"2024-08-01T04:16:15.841059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210445+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:16:16.210445+0000","last_peered":"2024-08-01T04:16:16.210445+0000","last_clean":"2024-08-01T04:16:16.210445+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:16:16.210445+0000","last_undegraded":"2024-08-01T04:16:16.210445+0000","last_fullsized":"2024-08-01T04:16:16.210445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210448+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:16:16.210448+0000","last_peered":"2024-08-01T04:16:16.210448+0000","last_clean":"2024-08-01T04:16:16.210448+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:16:16.210448+0000","last_undegraded":"2024-08-01T04:16:16.210448+0000","last_fullsized":"2024-08-01T04:16:16.210448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841282+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:16:15.841282+0000","last_peered":"2024-08-01T04:16:15.841282+0000","last_clean":"2024-08-01T04:16:15.841282+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:16:15.841282+0000","last_undegraded":"2024-08-01T04:16:15.841282+0000","last_fullsized":"2024-08-01T04:16:15.841282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880005+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:16:15.880005+0000","last_peered":"2024-08-01T04:16:15.880005+0000","last_clean":"2024-08-01T04:16:15.880005+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:16:15.880005+0000","last_undegraded":"2024-08-01T04:16:15.880005+0000","last_fullsized":"2024-08-01T04:16:15.880005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210288+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:16:16.210288+0000","last_peered":"2024-08-01T04:16:16.210288+0000","last_clean":"2024-08-01T04:16:16.210288+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:16:16.210288+0000","last_undegraded":"2024-08-01T04:16:16.210288+0000","last_fullsized":"2024-08-01T04:16:16.210288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877149+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:16:15.877149+0000","last_peered":"2024-08-01T04:16:15.877149+0000","last_clean":"2024-08-01T04:16:15.877149+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:16:15.877149+0000","last_undegraded":"2024-08-01T04:16:15.877149+0000","last_fullsized":"2024-08-01T04:16:15.877149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877440+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:16:15.877440+0000","last_peered":"2024-08-01T04:16:15.877440+0000","last_clean":"2024-08-01T04:16:15.877440+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:16:15.877440+0000","last_undegraded":"2024-08-01T04:16:15.877440+0000","last_fullsized":"2024-08-01T04:16:15.877440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963269+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:16:15.963269+0000","last_peered":"2024-08-01T04:16:15.963269+0000","last_clean":"2024-08-01T04:16:15.963269+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:16:15.963269+0000","last_undegraded":"2024-08-01T04:16:15.963269+0000","last_fullsized":"2024-08-01T04:16:15.963269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210461+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:16:16.210461+0000","last_peered":"2024-08-01T04:16:16.210461+0000","last_clean":"2024-08-01T04:16:16.210461+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:16:16.210461+0000","last_undegraded":"2024-08-01T04:16:16.210461+0000","last_fullsized":"2024-08-01T04:16:16.210461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967176+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:16:15.967176+0000","last_peered":"2024-08-01T04:16:15.967176+0000","last_clean":"2024-08-01T04:16:15.967176+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:16:15.967176+0000","last_undegraded":"2024-08-01T04:16:15.967176+0000","last_fullsized":"2024-08-01T04:16:15.967176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841074+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:16:15.841074+0000","last_peered":"2024-08-01T04:16:15.841074+0000","last_clean":"2024-08-01T04:16:15.841074+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:16:15.841074+0000","last_undegraded":"2024-08-01T04:16:15.841074+0000","last_fullsized":"2024-08-01T04:16:15.841074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967171+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:16:15.967171+0000","last_peered":"2024-08-01T04:16:15.967171+0000","last_clean":"2024-08-01T04:16:15.967171+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:16:15.967171+0000","last_undegraded":"2024-08-01T04:16:15.967171+0000","last_fullsized":"2024-08-01T04:16:15.967171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954025+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:16:15.954025+0000","last_peered":"2024-08-01T04:16:15.954025+0000","last_clean":"2024-08-01T04:16:15.954025+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:16:15.954025+0000","last_undegraded":"2024-08-01T04:16:15.954025+0000","last_fullsized":"2024-08-01T04:16:15.954025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877800+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:16:15.877800+0000","last_peered":"2024-08-01T04:16:15.877800+0000","last_clean":"2024-08-01T04:16:15.877800+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:16:15.877800+0000","last_undegraded":"2024-08-01T04:16:15.877800+0000","last_fullsized":"2024-08-01T04:16:15.877800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'48","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.172158+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:16:16.172158+0000","last_peered":"2024-08-01T04:16:16.172158+0000","last_clean":"2024-08-01T04:16:16.172158+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:16:16.172158+0000","last_undegraded":"2024-08-01T04:16:16.172158+0000","last_fullsized":"2024-08-01T04:16:16.172158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880558+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:16:15.880558+0000","last_peered":"2024-08-01T04:16:15.880558+0000","last_clean":"2024-08-01T04:16:15.880558+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:16:15.880558+0000","last_undegraded":"2024-08-01T04:16:15.880558+0000","last_fullsized":"2024-08-01T04:16:15.880558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841491+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:16:15.841491+0000","last_peered":"2024-08-01T04:16:15.841491+0000","last_clean":"2024-08-01T04:16:15.841491+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:16:15.841491+0000","last_undegraded":"2024-08-01T04:16:15.841491+0000","last_fullsized":"2024-08-01T04:16:15.841491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212996+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:16:16.212996+0000","last_peered":"2024-08-01T04:16:16.212996+0000","last_clean":"2024-08-01T04:16:16.212996+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:16:16.212996+0000","last_undegraded":"2024-08-01T04:16:16.212996+0000","last_fullsized":"2024-08-01T04:16:16.212996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005685+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:16:16.005685+0000","last_peered":"2024-08-01T04:16:16.005685+0000","last_clean":"2024-08-01T04:16:16.005685+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:16:16.005685+0000","last_undegraded":"2024-08-01T04:16:16.005685+0000","last_fullsized":"2024-08-01T04:16:16.005685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953515+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:16:15.953515+0000","last_peered":"2024-08-01T04:16:15.953515+0000","last_clean":"2024-08-01T04:16:15.953515+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:16:15.953515+0000","last_undegraded":"2024-08-01T04:16:15.953515+0000","last_fullsized":"2024-08-01T04:16:15.953515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879984+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:16:15.879984+0000","last_peered":"2024-08-01T04:16:15.879984+0000","last_clean":"2024-08-01T04:16:15.879984+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:16:15.879984+0000","last_undegraded":"2024-08-01T04:16:15.879984+0000","last_fullsized":"2024-08-01T04:16:15.879984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550822+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:16:16.550822+0000","last_peered":"2024-08-01T04:16:16.550822+0000","last_clean":"2024-08-01T04:16:16.550822+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:16:16.550822+0000","last_undegraded":"2024-08-01T04:16:16.550822+0000","last_fullsized":"2024-08-01T04:16:16.550822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550847+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:16:16.550847+0000","last_peered":"2024-08-01T04:16:16.550847+0000","last_clean":"2024-08-01T04:16:16.550847+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:16:16.550847+0000","last_undegraded":"2024-08-01T04:16:16.550847+0000","last_fullsized":"2024-08-01T04:16:16.550847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880233+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:16:15.880233+0000","last_peered":"2024-08-01T04:16:15.880233+0000","last_clean":"2024-08-01T04:16:15.880233+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:16:15.880233+0000","last_undegraded":"2024-08-01T04:16:15.880233+0000","last_fullsized":"2024-08-01T04:16:15.880233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953606+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:16:15.953606+0000","last_peered":"2024-08-01T04:16:15.953606+0000","last_clean":"2024-08-01T04:16:15.953606+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:16:15.953606+0000","last_undegraded":"2024-08-01T04:16:15.953606+0000","last_fullsized":"2024-08-01T04:16:15.953606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210241+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:16:16.210241+0000","last_peered":"2024-08-01T04:16:16.210241+0000","last_clean":"2024-08-01T04:16:16.210241+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:16:16.210241+0000","last_undegraded":"2024-08-01T04:16:16.210241+0000","last_fullsized":"2024-08-01T04:16:16.210241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963633+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:16:15.963633+0000","last_peered":"2024-08-01T04:16:15.963633+0000","last_clean":"2024-08-01T04:16:15.963633+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:16:15.963633+0000","last_undegraded":"2024-08-01T04:16:15.963633+0000","last_fullsized":"2024-08-01T04:16:15.963633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953850+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:16:15.953850+0000","last_peered":"2024-08-01T04:16:15.953850+0000","last_clean":"2024-08-01T04:16:15.953850+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:16:15.953850+0000","last_undegraded":"2024-08-01T04:16:15.953850+0000","last_fullsized":"2024-08-01T04:16:15.953850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005670+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:16:16.005670+0000","last_peered":"2024-08-01T04:16:16.005670+0000","last_clean":"2024-08-01T04:16:16.005670+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:16:16.005670+0000","last_undegraded":"2024-08-01T04:16:16.005670+0000","last_fullsized":"2024-08-01T04:16:16.005670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550806+0000","last_change":"2024-08-01T04:15:50.403853+0000","last_active":"2024-08-01T04:16:16.550806+0000","last_peered":"2024-08-01T04:16:16.550806+0000","last_clean":"2024-08-01T04:16:16.550806+0000","last_became_active":"2024-08-01T04:15:50.400282+0000","last_became_peered":"2024-08-01T04:15:50.400282+0000","last_unstale":"2024-08-01T04:16:16.550806+0000","last_undegraded":"2024-08-01T04:16:16.550806+0000","last_fullsized":"2024-08-01T04:16:16.550806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:11:52.649176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213209+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:16:16.213209+0000","last_peered":"2024-08-01T04:16:16.213209+0000","last_clean":"2024-08-01T04:16:16.213209+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:16:16.213209+0000","last_undegraded":"2024-08-01T04:16:16.213209+0000","last_fullsized":"2024-08-01T04:16:16.213209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880724+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:16:15.880724+0000","last_peered":"2024-08-01T04:16:15.880724+0000","last_clean":"2024-08-01T04:16:15.880724+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:16:15.880724+0000","last_undegraded":"2024-08-01T04:16:15.880724+0000","last_fullsized":"2024-08-01T04:16:15.880724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880592+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:16:15.880592+0000","last_peered":"2024-08-01T04:16:15.880592+0000","last_clean":"2024-08-01T04:16:15.880592+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:16:15.880592+0000","last_undegraded":"2024-08-01T04:16:15.880592+0000","last_fullsized":"2024-08-01T04:16:15.880592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840998+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:16:15.840998+0000","last_peered":"2024-08-01T04:16:15.840998+0000","last_clean":"2024-08-01T04:16:15.840998+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:16:15.840998+0000","last_undegraded":"2024-08-01T04:16:15.840998+0000","last_fullsized":"2024-08-01T04:16:15.840998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598921,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":891,"num_read_kb":556,"num_write":296,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1796763,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449129,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":484724,"data_compressed":11581,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678063,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1167360,"data_stored":1280269,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":887219,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103226,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":887139,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1077481,"data_compressed":47455,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":493392,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":1217667,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":494750,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":559751,"data_compressed":17833,"data_compressed_allocated":24576,"data_compressed_original":106496,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":557956,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":561838,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":494663,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133997,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202752,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:16:23.379 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-01T04:16:23.379 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-01T04:16:23.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:23 smithi003 ceph-mon[32262]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:23.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:23 smithi003 ceph-mon[32262]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:23.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:23 smithi003 ceph-mon[32262]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:23.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:23 smithi163 ceph-mon[37383]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:23 smithi163 ceph-mon[37383]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:23 smithi163 ceph-mon[37383]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 -- 172.21.15.3:0/2142039518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01077e0 msgr2=0x7fabe0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 --2- 172.21.15.3:0/2142039518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01077e0 0x7fabe0107bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009ca0 tx=0x7fabd002a8c0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 -- 172.21.15.3:0/2142039518 shutdown_connections 2024-08-01T04:16:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 --2- 172.21.15.3:0/2142039518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe0102170 0x7fabe010e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 --2- 172.21.15.3:0/2142039518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe01017d0 0x7fabe0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 --2- 172.21.15.3:0/2142039518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01077e0 0x7fabe0107bc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 -- 172.21.15.3:0/2142039518 >> 172.21.15.3:0/2142039518 conn(0x7fabe00fd4f0 msgr2=0x7fabe00ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 -- 172.21.15.3:0/2142039518 shutdown_connections 2024-08-01T04:16:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.628+0000 7fabe8200640 1 -- 172.21.15.3:0/2142039518 wait complete. 2024-08-01T04:16:23.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.629+0000 7fabe8200640 1 Processor -- start 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.629+0000 7fabe8200640 1 -- start start 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe8200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 0x7fabe019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe8200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe0102170 0x7fabe019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe8200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe01077e0 0x7fabe019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe8200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0111c70 con 0x7fabe0102170 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe8200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0111af0 con 0x7fabe01017d0 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe8200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0111df0 con 0x7fabe01077e0 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 0x7fabe019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 0x7fabe019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33748/0 (socket says 172.21.15.3:33748) 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe5800640 1 -- 172.21.15.3:0/4002150216 learned_addr learned my addr 172.21.15.3:0/4002150216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe4e00640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe0102170 0x7fabe019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.630+0000 7fabe6200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe01077e0 0x7fabe019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe6200640 1 -- 172.21.15.3:0/4002150216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 msgr2=0x7fabe019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe6200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 0x7fabe019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe6200640 1 -- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe0102170 msgr2=0x7fabe019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe6200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe0102170 0x7fabe019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe6200640 1 -- 172.21.15.3:0/4002150216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe019f310 con 0x7fabe01077e0 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe5800640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 0x7fabe019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe4e00640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe0102170 0x7fabe019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe6200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe01077e0 0x7fabe019ebe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e970 tx=0x7fabd800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd800ccc0 con 0x7fabe01077e0 2024-08-01T04:16:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd8005020 con 0x7fabe01077e0 2024-08-01T04:16:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe019f600 con 0x7fabe01077e0 2024-08-01T04:16:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8004a00 con 0x7fabe01077e0 2024-08-01T04:16:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.631+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a6e90 con 0x7fabe01077e0 2024-08-01T04:16:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.633+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe0101c30 con 0x7fabe01077e0 2024-08-01T04:16:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.634+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7fabd80051d0 con 0x7fabe01077e0 2024-08-01T04:16:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.636+0000 7fabde000640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fabc40a2ea0 0x7fabc40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.636+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fabd80cf660 con 0x7fabe01077e0 2024-08-01T04:16:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.636+0000 7fabe5800640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fabc40a2ea0 0x7fabc40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.637+0000 7fabe5800640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fabc40a2ea0 0x7fabc40a5360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fabd0033040 tx=0x7fabd0005d80 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.640+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fabd809bf90 con 0x7fabe01077e0 2024-08-01T04:16:23.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:23 smithi160 ceph-mon[37224]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-01T04:16:23.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:23 smithi160 ceph-mon[37224]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-08-01T04:16:23.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:23 smithi160 ceph-mon[37224]: Updating smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/config/ceph.client.admin.keyring 2024-08-01T04:16:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.848+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabe00ff2c0 con 0x7fabc40a2ea0 2024-08-01T04:16:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.857+0000 7fabde000640 1 -- 172.21.15.3:0/4002150216 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258058 (secure 0 0 0) 0x7fabe00ff2c0 con 0x7fabc40a2ea0 2024-08-01T04:16:23.863 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:23.864 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fabc40a2ea0 msgr2=0x7fabc40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fabc40a2ea0 0x7fabc40a5360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fabd0033040 tx=0x7fabd0005d80 comp rx=0 tx=0).stop 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe01077e0 msgr2=0x7fabe019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe01077e0 0x7fabe019ebe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fabd800e970 tx=0x7fabd800ee30 comp rx=0 tx=0).stop 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 shutdown_connections 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fabc40a2ea0 0x7fabc40a5360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabe01077e0 0x7fabe019ebe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe0102170 0x7fabe019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 --2- 172.21.15.3:0/4002150216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabe01017d0 0x7fabe019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.862+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 >> 172.21.15.3:0/4002150216 conn(0x7fabe00fd4f0 msgr2=0x7fabe00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.863+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 shutdown_connections 2024-08-01T04:16:23.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:23.863+0000 7fabe8200640 1 -- 172.21.15.3:0/4002150216 wait complete. 2024-08-01T04:16:23.887 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-08-01T04:16:21.864911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189523,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1027,"num_read_kb":675,"num_write":479,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365592,"kb_used_data":12232,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576232,"statfs":{"total":1151940427776,"available":1151566061568,"internally_reserved":0,"allocated":12525568,"data_stored":8996849,"data_compressed":329658,"data_compressed_allocated":442368,"data_compressed_original":3342336,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.983444"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551165+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:16:16.551165+0000","last_peered":"2024-08-01T04:16:16.551165+0000","last_clean":"2024-08-01T04:16:16.551165+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:16:16.551165+0000","last_undegraded":"2024-08-01T04:16:16.551165+0000","last_fullsized":"2024-08-01T04:16:16.551165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212873+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:16:16.212873+0000","last_peered":"2024-08-01T04:16:16.212873+0000","last_clean":"2024-08-01T04:16:16.212873+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:16:16.212873+0000","last_undegraded":"2024-08-01T04:16:16.212873+0000","last_fullsized":"2024-08-01T04:16:16.212873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212612+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:16:16.212612+0000","last_peered":"2024-08-01T04:16:16.212612+0000","last_clean":"2024-08-01T04:16:16.212612+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:16:16.212612+0000","last_undegraded":"2024-08-01T04:16:16.212612+0000","last_fullsized":"2024-08-01T04:16:16.212612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840852+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:16:15.840852+0000","last_peered":"2024-08-01T04:16:15.840852+0000","last_clean":"2024-08-01T04:16:15.840852+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:16:15.840852+0000","last_undegraded":"2024-08-01T04:16:15.840852+0000","last_fullsized":"2024-08-01T04:16:15.840852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953626+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:16:15.953626+0000","last_peered":"2024-08-01T04:16:15.953626+0000","last_clean":"2024-08-01T04:16:15.953626+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:16:15.953626+0000","last_undegraded":"2024-08-01T04:16:15.953626+0000","last_fullsized":"2024-08-01T04:16:15.953626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.966981+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:16:15.966981+0000","last_peered":"2024-08-01T04:16:15.966981+0000","last_clean":"2024-08-01T04:16:15.966981+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:16:15.966981+0000","last_undegraded":"2024-08-01T04:16:15.966981+0000","last_fullsized":"2024-08-01T04:16:15.966981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210071+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:16:16.210071+0000","last_peered":"2024-08-01T04:16:16.210071+0000","last_clean":"2024-08-01T04:16:16.210071+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:16:16.210071+0000","last_undegraded":"2024-08-01T04:16:16.210071+0000","last_fullsized":"2024-08-01T04:16:16.210071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879729+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:16:15.879729+0000","last_peered":"2024-08-01T04:16:15.879729+0000","last_clean":"2024-08-01T04:16:15.879729+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:16:15.879729+0000","last_undegraded":"2024-08-01T04:16:15.879729+0000","last_fullsized":"2024-08-01T04:16:15.879729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006303+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:16:16.006303+0000","last_peered":"2024-08-01T04:16:16.006303+0000","last_clean":"2024-08-01T04:16:16.006303+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:16:16.006303+0000","last_undegraded":"2024-08-01T04:16:16.006303+0000","last_fullsized":"2024-08-01T04:16:16.006303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967373+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:16:15.967373+0000","last_peered":"2024-08-01T04:16:15.967373+0000","last_clean":"2024-08-01T04:16:15.967373+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:16:15.967373+0000","last_undegraded":"2024-08-01T04:16:15.967373+0000","last_fullsized":"2024-08-01T04:16:15.967373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210488+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:16:16.210488+0000","last_peered":"2024-08-01T04:16:16.210488+0000","last_clean":"2024-08-01T04:16:16.210488+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:16:16.210488+0000","last_undegraded":"2024-08-01T04:16:16.210488+0000","last_fullsized":"2024-08-01T04:16:16.210488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841219+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:16:15.841219+0000","last_peered":"2024-08-01T04:16:15.841219+0000","last_clean":"2024-08-01T04:16:15.841219+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:16:15.841219+0000","last_undegraded":"2024-08-01T04:16:15.841219+0000","last_fullsized":"2024-08-01T04:16:15.841219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877759+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:16:15.877759+0000","last_peered":"2024-08-01T04:16:15.877759+0000","last_clean":"2024-08-01T04:16:15.877759+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:16:15.877759+0000","last_undegraded":"2024-08-01T04:16:15.877759+0000","last_fullsized":"2024-08-01T04:16:15.877759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211836+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:16:16.211836+0000","last_peered":"2024-08-01T04:16:16.211836+0000","last_clean":"2024-08-01T04:16:16.211836+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:16:16.211836+0000","last_undegraded":"2024-08-01T04:16:16.211836+0000","last_fullsized":"2024-08-01T04:16:16.211836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211595+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:16:16.211595+0000","last_peered":"2024-08-01T04:16:16.211595+0000","last_clean":"2024-08-01T04:16:16.211595+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:16:16.211595+0000","last_undegraded":"2024-08-01T04:16:16.211595+0000","last_fullsized":"2024-08-01T04:16:16.211595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005632+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:16:16.005632+0000","last_peered":"2024-08-01T04:16:16.005632+0000","last_clean":"2024-08-01T04:16:16.005632+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:16:16.005632+0000","last_undegraded":"2024-08-01T04:16:16.005632+0000","last_fullsized":"2024-08-01T04:16:16.005632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551271+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:16:16.551271+0000","last_peered":"2024-08-01T04:16:16.551271+0000","last_clean":"2024-08-01T04:16:16.551271+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:16:16.551271+0000","last_undegraded":"2024-08-01T04:16:16.551271+0000","last_fullsized":"2024-08-01T04:16:16.551271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840840+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:16:15.840840+0000","last_peered":"2024-08-01T04:16:15.840840+0000","last_clean":"2024-08-01T04:16:15.840840+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:16:15.840840+0000","last_undegraded":"2024-08-01T04:16:15.840840+0000","last_fullsized":"2024-08-01T04:16:15.840840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840916+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:16:15.840916+0000","last_peered":"2024-08-01T04:16:15.840916+0000","last_clean":"2024-08-01T04:16:15.840916+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:16:15.840916+0000","last_undegraded":"2024-08-01T04:16:15.840916+0000","last_fullsized":"2024-08-01T04:16:15.840916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550965+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:16:16.550965+0000","last_peered":"2024-08-01T04:16:16.550965+0000","last_clean":"2024-08-01T04:16:16.550965+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:16:16.550965+0000","last_undegraded":"2024-08-01T04:16:16.550965+0000","last_fullsized":"2024-08-01T04:16:16.550965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877859+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:16:15.877859+0000","last_peered":"2024-08-01T04:16:15.877859+0000","last_clean":"2024-08-01T04:16:15.877859+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:16:15.877859+0000","last_undegraded":"2024-08-01T04:16:15.877859+0000","last_fullsized":"2024-08-01T04:16:15.877859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006020+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:16:16.006020+0000","last_peered":"2024-08-01T04:16:16.006020+0000","last_clean":"2024-08-01T04:16:16.006020+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:16:16.006020+0000","last_undegraded":"2024-08-01T04:16:16.006020+0000","last_fullsized":"2024-08-01T04:16:16.006020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879660+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:16:15.879660+0000","last_peered":"2024-08-01T04:16:15.879660+0000","last_clean":"2024-08-01T04:16:15.879660+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:16:15.879660+0000","last_undegraded":"2024-08-01T04:16:15.879660+0000","last_fullsized":"2024-08-01T04:16:15.879660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953511+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:16:15.953511+0000","last_peered":"2024-08-01T04:16:15.953511+0000","last_clean":"2024-08-01T04:16:15.953511+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:16:15.953511+0000","last_undegraded":"2024-08-01T04:16:15.953511+0000","last_fullsized":"2024-08-01T04:16:15.953511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880212+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:16:15.880212+0000","last_peered":"2024-08-01T04:16:15.880212+0000","last_clean":"2024-08-01T04:16:15.880212+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:16:15.880212+0000","last_undegraded":"2024-08-01T04:16:15.880212+0000","last_fullsized":"2024-08-01T04:16:15.880212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840932+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:16:15.840932+0000","last_peered":"2024-08-01T04:16:15.840932+0000","last_clean":"2024-08-01T04:16:15.840932+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:16:15.840932+0000","last_undegraded":"2024-08-01T04:16:15.840932+0000","last_fullsized":"2024-08-01T04:16:15.840932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551039+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:16:16.551039+0000","last_peered":"2024-08-01T04:16:16.551039+0000","last_clean":"2024-08-01T04:16:16.551039+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:16:16.551039+0000","last_undegraded":"2024-08-01T04:16:16.551039+0000","last_fullsized":"2024-08-01T04:16:16.551039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877756+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:16:15.877756+0000","last_peered":"2024-08-01T04:16:15.877756+0000","last_clean":"2024-08-01T04:16:15.877756+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:16:15.877756+0000","last_undegraded":"2024-08-01T04:16:15.877756+0000","last_fullsized":"2024-08-01T04:16:15.877756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212112+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:16:16.212112+0000","last_peered":"2024-08-01T04:16:16.212112+0000","last_clean":"2024-08-01T04:16:16.212112+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:16:16.212112+0000","last_undegraded":"2024-08-01T04:16:16.212112+0000","last_fullsized":"2024-08-01T04:16:16.212112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840499+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:16:15.840499+0000","last_peered":"2024-08-01T04:16:15.840499+0000","last_clean":"2024-08-01T04:16:15.840499+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:16:15.840499+0000","last_undegraded":"2024-08-01T04:16:15.840499+0000","last_fullsized":"2024-08-01T04:16:15.840499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840592+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:16:15.840592+0000","last_peered":"2024-08-01T04:16:15.840592+0000","last_clean":"2024-08-01T04:16:15.840592+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:16:15.840592+0000","last_undegraded":"2024-08-01T04:16:15.840592+0000","last_fullsized":"2024-08-01T04:16:15.840592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550743+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:16:16.550743+0000","last_peered":"2024-08-01T04:16:16.550743+0000","last_clean":"2024-08-01T04:16:16.550743+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:16:16.550743+0000","last_undegraded":"2024-08-01T04:16:16.550743+0000","last_fullsized":"2024-08-01T04:16:16.550743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953539+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:16:15.953539+0000","last_peered":"2024-08-01T04:16:15.953539+0000","last_clean":"2024-08-01T04:16:15.953539+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:16:15.953539+0000","last_undegraded":"2024-08-01T04:16:15.953539+0000","last_fullsized":"2024-08-01T04:16:15.953539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":173,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.834559+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:16:16.834559+0000","last_peered":"2024-08-01T04:16:16.834559+0000","last_clean":"2024-08-01T04:16:16.834559+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:16:16.834559+0000","last_undegraded":"2024-08-01T04:16:16.834559+0000","last_fullsized":"2024-08-01T04:16:16.834559+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963413+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:16:15.963413+0000","last_peered":"2024-08-01T04:16:15.963413+0000","last_clean":"2024-08-01T04:16:15.963413+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:16:15.963413+0000","last_undegraded":"2024-08-01T04:16:15.963413+0000","last_fullsized":"2024-08-01T04:16:15.963413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967038+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:16:15.967038+0000","last_peered":"2024-08-01T04:16:15.967038+0000","last_clean":"2024-08-01T04:16:15.967038+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:16:15.967038+0000","last_undegraded":"2024-08-01T04:16:15.967038+0000","last_fullsized":"2024-08-01T04:16:15.967038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877210+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:16:15.877210+0000","last_peered":"2024-08-01T04:16:15.877210+0000","last_clean":"2024-08-01T04:16:15.877210+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:16:15.877210+0000","last_undegraded":"2024-08-01T04:16:15.877210+0000","last_fullsized":"2024-08-01T04:16:15.877210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879844+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:16:15.879844+0000","last_peered":"2024-08-01T04:16:15.879844+0000","last_clean":"2024-08-01T04:16:15.879844+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:16:15.879844+0000","last_undegraded":"2024-08-01T04:16:15.879844+0000","last_fullsized":"2024-08-01T04:16:15.879844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963382+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:16:15.963382+0000","last_peered":"2024-08-01T04:16:15.963382+0000","last_clean":"2024-08-01T04:16:15.963382+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:16:15.963382+0000","last_undegraded":"2024-08-01T04:16:15.963382+0000","last_fullsized":"2024-08-01T04:16:15.963382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840510+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:16:15.840510+0000","last_peered":"2024-08-01T04:16:15.840510+0000","last_clean":"2024-08-01T04:16:15.840510+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:16:15.840510+0000","last_undegraded":"2024-08-01T04:16:15.840510+0000","last_fullsized":"2024-08-01T04:16:15.840510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.834039+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:16:16.834039+0000","last_peered":"2024-08-01T04:16:16.834039+0000","last_clean":"2024-08-01T04:16:16.834039+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:16:16.834039+0000","last_undegraded":"2024-08-01T04:16:16.834039+0000","last_fullsized":"2024-08-01T04:16:16.834039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006102+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:16:16.006102+0000","last_peered":"2024-08-01T04:16:16.006102+0000","last_clean":"2024-08-01T04:16:16.006102+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:16:16.006102+0000","last_undegraded":"2024-08-01T04:16:16.006102+0000","last_fullsized":"2024-08-01T04:16:16.006102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210517+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:16:16.210517+0000","last_peered":"2024-08-01T04:16:16.210517+0000","last_clean":"2024-08-01T04:16:16.210517+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:16:16.210517+0000","last_undegraded":"2024-08-01T04:16:16.210517+0000","last_fullsized":"2024-08-01T04:16:16.210517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213346+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:16:16.213346+0000","last_peered":"2024-08-01T04:16:16.213346+0000","last_clean":"2024-08-01T04:16:16.213346+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:16:16.213346+0000","last_undegraded":"2024-08-01T04:16:16.213346+0000","last_fullsized":"2024-08-01T04:16:16.213346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963763+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:16:15.963763+0000","last_peered":"2024-08-01T04:16:15.963763+0000","last_clean":"2024-08-01T04:16:15.963763+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:16:15.963763+0000","last_undegraded":"2024-08-01T04:16:15.963763+0000","last_fullsized":"2024-08-01T04:16:15.963763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005853+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:16:16.005853+0000","last_peered":"2024-08-01T04:16:16.005853+0000","last_clean":"2024-08-01T04:16:16.005853+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:16:16.005853+0000","last_undegraded":"2024-08-01T04:16:16.005853+0000","last_fullsized":"2024-08-01T04:16:16.005853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":333,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.835162+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:16:16.835162+0000","last_peered":"2024-08-01T04:16:16.835162+0000","last_clean":"2024-08-01T04:16:16.835162+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:16:16.835162+0000","last_undegraded":"2024-08-01T04:16:16.835162+0000","last_fullsized":"2024-08-01T04:16:16.835162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68653,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":266,"num_read_kb":199,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953989+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:16:15.953989+0000","last_peered":"2024-08-01T04:16:15.953989+0000","last_clean":"2024-08-01T04:16:15.953989+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:16:15.953989+0000","last_undegraded":"2024-08-01T04:16:15.953989+0000","last_fullsized":"2024-08-01T04:16:15.953989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840839+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:16:15.840839+0000","last_peered":"2024-08-01T04:16:15.840839+0000","last_clean":"2024-08-01T04:16:15.840839+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:16:15.840839+0000","last_undegraded":"2024-08-01T04:16:15.840839+0000","last_fullsized":"2024-08-01T04:16:15.840839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877540+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:16:15.877540+0000","last_peered":"2024-08-01T04:16:15.877540+0000","last_clean":"2024-08-01T04:16:15.877540+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:16:15.877540+0000","last_undegraded":"2024-08-01T04:16:15.877540+0000","last_fullsized":"2024-08-01T04:16:15.877540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963535+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:16:15.963535+0000","last_peered":"2024-08-01T04:16:15.963535+0000","last_clean":"2024-08-01T04:16:15.963535+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:16:15.963535+0000","last_undegraded":"2024-08-01T04:16:15.963535+0000","last_fullsized":"2024-08-01T04:16:15.963535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210232+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:16:16.210232+0000","last_peered":"2024-08-01T04:16:16.210232+0000","last_clean":"2024-08-01T04:16:16.210232+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:16:16.210232+0000","last_undegraded":"2024-08-01T04:16:16.210232+0000","last_fullsized":"2024-08-01T04:16:16.210232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.259331+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:16:16.259331+0000","last_peered":"2024-08-01T04:16:16.259331+0000","last_clean":"2024-08-01T04:16:16.259331+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:16:16.259331+0000","last_undegraded":"2024-08-01T04:16:16.259331+0000","last_fullsized":"2024-08-01T04:16:16.259331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210072+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:16:16.210072+0000","last_peered":"2024-08-01T04:16:16.210072+0000","last_clean":"2024-08-01T04:16:16.210072+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:16:16.210072+0000","last_undegraded":"2024-08-01T04:16:16.210072+0000","last_fullsized":"2024-08-01T04:16:16.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953840+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:16:15.953840+0000","last_peered":"2024-08-01T04:16:15.953840+0000","last_clean":"2024-08-01T04:16:15.953840+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:16:15.953840+0000","last_undegraded":"2024-08-01T04:16:15.953840+0000","last_fullsized":"2024-08-01T04:16:15.953840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963507+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:16:15.963507+0000","last_peered":"2024-08-01T04:16:15.963507+0000","last_clean":"2024-08-01T04:16:15.963507+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:16:15.963507+0000","last_undegraded":"2024-08-01T04:16:15.963507+0000","last_fullsized":"2024-08-01T04:16:15.963507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954076+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:16:15.954076+0000","last_peered":"2024-08-01T04:16:15.954076+0000","last_clean":"2024-08-01T04:16:15.954076+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:16:15.954076+0000","last_undegraded":"2024-08-01T04:16:15.954076+0000","last_fullsized":"2024-08-01T04:16:15.954076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551119+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:16:16.551119+0000","last_peered":"2024-08-01T04:16:16.551119+0000","last_clean":"2024-08-01T04:16:16.551119+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:16:16.551119+0000","last_undegraded":"2024-08-01T04:16:16.551119+0000","last_fullsized":"2024-08-01T04:16:16.551119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877775+0000","last_change":"2024-08-01T04:15:50.404577+0000","last_active":"2024-08-01T04:16:15.877775+0000","last_peered":"2024-08-01T04:16:15.877775+0000","last_clean":"2024-08-01T04:16:15.877775+0000","last_became_active":"2024-08-01T04:15:50.400018+0000","last_became_peered":"2024-08-01T04:15:50.400018+0000","last_unstale":"2024-08-01T04:16:15.877775+0000","last_undegraded":"2024-08-01T04:16:15.877775+0000","last_fullsized":"2024-08-01T04:16:15.877775+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:22:39.045462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210494+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:16:16.210494+0000","last_peered":"2024-08-01T04:16:16.210494+0000","last_clean":"2024-08-01T04:16:16.210494+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:16:16.210494+0000","last_undegraded":"2024-08-01T04:16:16.210494+0000","last_fullsized":"2024-08-01T04:16:16.210494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'39","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.163057+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:16:16.163057+0000","last_peered":"2024-08-01T04:16:16.163057+0000","last_clean":"2024-08-01T04:16:16.163057+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:16:16.163057+0000","last_undegraded":"2024-08-01T04:16:16.163057+0000","last_fullsized":"2024-08-01T04:16:16.163057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210202+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:16:16.210202+0000","last_peered":"2024-08-01T04:16:16.210202+0000","last_clean":"2024-08-01T04:16:16.210202+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:16:16.210202+0000","last_undegraded":"2024-08-01T04:16:16.210202+0000","last_fullsized":"2024-08-01T04:16:16.210202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841059+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:16:15.841059+0000","last_peered":"2024-08-01T04:16:15.841059+0000","last_clean":"2024-08-01T04:16:15.841059+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:16:15.841059+0000","last_undegraded":"2024-08-01T04:16:15.841059+0000","last_fullsized":"2024-08-01T04:16:15.841059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210445+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:16:16.210445+0000","last_peered":"2024-08-01T04:16:16.210445+0000","last_clean":"2024-08-01T04:16:16.210445+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:16:16.210445+0000","last_undegraded":"2024-08-01T04:16:16.210445+0000","last_fullsized":"2024-08-01T04:16:16.210445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210448+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:16:16.210448+0000","last_peered":"2024-08-01T04:16:16.210448+0000","last_clean":"2024-08-01T04:16:16.210448+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:16:16.210448+0000","last_undegraded":"2024-08-01T04:16:16.210448+0000","last_fullsized":"2024-08-01T04:16:16.210448+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841282+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:16:15.841282+0000","last_peered":"2024-08-01T04:16:15.841282+0000","last_clean":"2024-08-01T04:16:15.841282+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:16:15.841282+0000","last_undegraded":"2024-08-01T04:16:15.841282+0000","last_fullsized":"2024-08-01T04:16:15.841282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880005+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:16:15.880005+0000","last_peered":"2024-08-01T04:16:15.880005+0000","last_clean":"2024-08-01T04:16:15.880005+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:16:15.880005+0000","last_undegraded":"2024-08-01T04:16:15.880005+0000","last_fullsized":"2024-08-01T04:16:15.880005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210288+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:16:16.210288+0000","last_peered":"2024-08-01T04:16:16.210288+0000","last_clean":"2024-08-01T04:16:16.210288+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:16:16.210288+0000","last_undegraded":"2024-08-01T04:16:16.210288+0000","last_fullsized":"2024-08-01T04:16:16.210288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877149+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:16:15.877149+0000","last_peered":"2024-08-01T04:16:15.877149+0000","last_clean":"2024-08-01T04:16:15.877149+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:16:15.877149+0000","last_undegraded":"2024-08-01T04:16:15.877149+0000","last_fullsized":"2024-08-01T04:16:15.877149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877440+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:16:15.877440+0000","last_peered":"2024-08-01T04:16:15.877440+0000","last_clean":"2024-08-01T04:16:15.877440+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:16:15.877440+0000","last_undegraded":"2024-08-01T04:16:15.877440+0000","last_fullsized":"2024-08-01T04:16:15.877440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963269+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:16:15.963269+0000","last_peered":"2024-08-01T04:16:15.963269+0000","last_clean":"2024-08-01T04:16:15.963269+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:16:15.963269+0000","last_undegraded":"2024-08-01T04:16:15.963269+0000","last_fullsized":"2024-08-01T04:16:15.963269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210461+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:16:16.210461+0000","last_peered":"2024-08-01T04:16:16.210461+0000","last_clean":"2024-08-01T04:16:16.210461+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:16:16.210461+0000","last_undegraded":"2024-08-01T04:16:16.210461+0000","last_fullsized":"2024-08-01T04:16:16.210461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967176+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:16:15.967176+0000","last_peered":"2024-08-01T04:16:15.967176+0000","last_clean":"2024-08-01T04:16:15.967176+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:16:15.967176+0000","last_undegraded":"2024-08-01T04:16:15.967176+0000","last_fullsized":"2024-08-01T04:16:15.967176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841074+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:16:15.841074+0000","last_peered":"2024-08-01T04:16:15.841074+0000","last_clean":"2024-08-01T04:16:15.841074+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:16:15.841074+0000","last_undegraded":"2024-08-01T04:16:15.841074+0000","last_fullsized":"2024-08-01T04:16:15.841074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967171+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:16:15.967171+0000","last_peered":"2024-08-01T04:16:15.967171+0000","last_clean":"2024-08-01T04:16:15.967171+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:16:15.967171+0000","last_undegraded":"2024-08-01T04:16:15.967171+0000","last_fullsized":"2024-08-01T04:16:15.967171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954025+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:16:15.954025+0000","last_peered":"2024-08-01T04:16:15.954025+0000","last_clean":"2024-08-01T04:16:15.954025+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:16:15.954025+0000","last_undegraded":"2024-08-01T04:16:15.954025+0000","last_fullsized":"2024-08-01T04:16:15.954025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877800+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:16:15.877800+0000","last_peered":"2024-08-01T04:16:15.877800+0000","last_clean":"2024-08-01T04:16:15.877800+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:16:15.877800+0000","last_undegraded":"2024-08-01T04:16:15.877800+0000","last_fullsized":"2024-08-01T04:16:15.877800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'48","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.172158+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:16:16.172158+0000","last_peered":"2024-08-01T04:16:16.172158+0000","last_clean":"2024-08-01T04:16:16.172158+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:16:16.172158+0000","last_undegraded":"2024-08-01T04:16:16.172158+0000","last_fullsized":"2024-08-01T04:16:16.172158+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880558+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:16:15.880558+0000","last_peered":"2024-08-01T04:16:15.880558+0000","last_clean":"2024-08-01T04:16:15.880558+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:16:15.880558+0000","last_undegraded":"2024-08-01T04:16:15.880558+0000","last_fullsized":"2024-08-01T04:16:15.880558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841491+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:16:15.841491+0000","last_peered":"2024-08-01T04:16:15.841491+0000","last_clean":"2024-08-01T04:16:15.841491+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:16:15.841491+0000","last_undegraded":"2024-08-01T04:16:15.841491+0000","last_fullsized":"2024-08-01T04:16:15.841491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212996+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:16:16.212996+0000","last_peered":"2024-08-01T04:16:16.212996+0000","last_clean":"2024-08-01T04:16:16.212996+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:16:16.212996+0000","last_undegraded":"2024-08-01T04:16:16.212996+0000","last_fullsized":"2024-08-01T04:16:16.212996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005685+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:16:16.005685+0000","last_peered":"2024-08-01T04:16:16.005685+0000","last_clean":"2024-08-01T04:16:16.005685+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:16:16.005685+0000","last_undegraded":"2024-08-01T04:16:16.005685+0000","last_fullsized":"2024-08-01T04:16:16.005685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953515+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:16:15.953515+0000","last_peered":"2024-08-01T04:16:15.953515+0000","last_clean":"2024-08-01T04:16:15.953515+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:16:15.953515+0000","last_undegraded":"2024-08-01T04:16:15.953515+0000","last_fullsized":"2024-08-01T04:16:15.953515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879984+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:16:15.879984+0000","last_peered":"2024-08-01T04:16:15.879984+0000","last_clean":"2024-08-01T04:16:15.879984+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:16:15.879984+0000","last_undegraded":"2024-08-01T04:16:15.879984+0000","last_fullsized":"2024-08-01T04:16:15.879984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550822+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:16:16.550822+0000","last_peered":"2024-08-01T04:16:16.550822+0000","last_clean":"2024-08-01T04:16:16.550822+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:16:16.550822+0000","last_undegraded":"2024-08-01T04:16:16.550822+0000","last_fullsized":"2024-08-01T04:16:16.550822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550847+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:16:16.550847+0000","last_peered":"2024-08-01T04:16:16.550847+0000","last_clean":"2024-08-01T04:16:16.550847+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:16:16.550847+0000","last_undegraded":"2024-08-01T04:16:16.550847+0000","last_fullsized":"2024-08-01T04:16:16.550847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880233+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:16:15.880233+0000","last_peered":"2024-08-01T04:16:15.880233+0000","last_clean":"2024-08-01T04:16:15.880233+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:16:15.880233+0000","last_undegraded":"2024-08-01T04:16:15.880233+0000","last_fullsized":"2024-08-01T04:16:15.880233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953606+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:16:15.953606+0000","last_peered":"2024-08-01T04:16:15.953606+0000","last_clean":"2024-08-01T04:16:15.953606+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:16:15.953606+0000","last_undegraded":"2024-08-01T04:16:15.953606+0000","last_fullsized":"2024-08-01T04:16:15.953606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210241+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:16:16.210241+0000","last_peered":"2024-08-01T04:16:16.210241+0000","last_clean":"2024-08-01T04:16:16.210241+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:16:16.210241+0000","last_undegraded":"2024-08-01T04:16:16.210241+0000","last_fullsized":"2024-08-01T04:16:16.210241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963633+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:16:15.963633+0000","last_peered":"2024-08-01T04:16:15.963633+0000","last_clean":"2024-08-01T04:16:15.963633+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:16:15.963633+0000","last_undegraded":"2024-08-01T04:16:15.963633+0000","last_fullsized":"2024-08-01T04:16:15.963633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953850+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:16:15.953850+0000","last_peered":"2024-08-01T04:16:15.953850+0000","last_clean":"2024-08-01T04:16:15.953850+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:16:15.953850+0000","last_undegraded":"2024-08-01T04:16:15.953850+0000","last_fullsized":"2024-08-01T04:16:15.953850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005670+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:16:16.005670+0000","last_peered":"2024-08-01T04:16:16.005670+0000","last_clean":"2024-08-01T04:16:16.005670+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:16:16.005670+0000","last_undegraded":"2024-08-01T04:16:16.005670+0000","last_fullsized":"2024-08-01T04:16:16.005670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550806+0000","last_change":"2024-08-01T04:15:50.403853+0000","last_active":"2024-08-01T04:16:16.550806+0000","last_peered":"2024-08-01T04:16:16.550806+0000","last_clean":"2024-08-01T04:16:16.550806+0000","last_became_active":"2024-08-01T04:15:50.400282+0000","last_became_peered":"2024-08-01T04:15:50.400282+0000","last_unstale":"2024-08-01T04:16:16.550806+0000","last_undegraded":"2024-08-01T04:16:16.550806+0000","last_fullsized":"2024-08-01T04:16:16.550806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:11:52.649176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213209+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:16:16.213209+0000","last_peered":"2024-08-01T04:16:16.213209+0000","last_clean":"2024-08-01T04:16:16.213209+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:16:16.213209+0000","last_undegraded":"2024-08-01T04:16:16.213209+0000","last_fullsized":"2024-08-01T04:16:16.213209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880724+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:16:15.880724+0000","last_peered":"2024-08-01T04:16:15.880724+0000","last_clean":"2024-08-01T04:16:15.880724+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:16:15.880724+0000","last_undegraded":"2024-08-01T04:16:15.880724+0000","last_fullsized":"2024-08-01T04:16:15.880724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880592+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:16:15.880592+0000","last_peered":"2024-08-01T04:16:15.880592+0000","last_clean":"2024-08-01T04:16:15.880592+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:16:15.880592+0000","last_undegraded":"2024-08-01T04:16:15.880592+0000","last_fullsized":"2024-08-01T04:16:15.880592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840998+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:16:15.840998+0000","last_peered":"2024-08-01T04:16:15.840998+0000","last_clean":"2024-08-01T04:16:15.840998+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:16:15.840998+0000","last_undegraded":"2024-08-01T04:16:15.840998+0000","last_fullsized":"2024-08-01T04:16:15.840998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598921,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":891,"num_read_kb":556,"num_write":296,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1796763,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449129,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":484724,"data_compressed":11581,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678063,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1167360,"data_stored":1280269,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":887219,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103226,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":887139,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1077481,"data_compressed":47455,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":493392,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1089536,"data_stored":1217667,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":494750,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":559751,"data_compressed":17833,"data_compressed_allocated":24576,"data_compressed_original":106496,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":557956,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":561838,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":494663,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133997,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":202752,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:16:23.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:16:24.423 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:24.547 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:24.548 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:24.548 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:24 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:24.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:24.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:24.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:24.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:24 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:24.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:24.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:24 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.067+0000 7fd12d200640 1 -- 172.21.15.3:0/4126049462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281016f0 msgr2=0x7fd128101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.067+0000 7fd12d200640 1 --2- 172.21.15.3:0/4126049462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281016f0 0x7fd128101b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd110009930 tx=0x7fd11002a440 comp rx=0 tx=0).stop 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 -- 172.21.15.3:0/4126049462 shutdown_connections 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 --2- 172.21.15.3:0/4126049462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd128102090 0x7fd12810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 --2- 172.21.15.3:0/4126049462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281016f0 0x7fd128101b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 --2- 172.21.15.3:0/4126049462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1281076f0 0x7fd128107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 -- 172.21.15.3:0/4126049462 >> 172.21.15.3:0/4126049462 conn(0x7fd1280fd470 msgr2=0x7fd1280ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 -- 172.21.15.3:0/4126049462 shutdown_connections 2024-08-01T04:16:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.068+0000 7fd12d200640 1 -- 172.21.15.3:0/4126049462 wait complete. 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 Processor -- start 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 -- start start 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1281016f0 0x7fd128074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd128102090 0x7fd128074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 0x7fd128074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111af0 con 0x7fd1281076f0 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111970 con 0x7fd128102090 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.069+0000 7fd12d200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd128111c70 con 0x7fd1281016f0 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 0x7fd128074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 0x7fd128074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43624/0 (socket says 172.21.15.3:43624) 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 -- 172.21.15.3:0/3850596394 learned_addr learned my addr 172.21.15.3:0/3850596394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:25.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127400640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1281016f0 0x7fd128074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd126a00640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd128102090 0x7fd128074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 -- 172.21.15.3:0/3850596394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1281016f0 msgr2=0x7fd128074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1281016f0 0x7fd128074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 -- 172.21.15.3:0/3850596394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd128102090 msgr2=0x7fd128074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd128102090 0x7fd128074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.070+0000 7fd127e00640 1 -- 172.21.15.3:0/3850596394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1281a64b0 con 0x7fd1281076f0 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.071+0000 7fd127e00640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 0x7fd128074ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e970 tx=0x7fd12000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.071+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12000ccc0 con 0x7fd1281076f0 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.071+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12000ce60 con 0x7fd1281076f0 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.071+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120010490 con 0x7fd1281076f0 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.071+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1281a67a0 con 0x7fd1281076f0 2024-08-01T04:16:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.071+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a6d00 con 0x7fd1281076f0 2024-08-01T04:16:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.072+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd128101b50 con 0x7fd1281076f0 2024-08-01T04:16:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.075+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7fd120005000 con 0x7fd1281076f0 2024-08-01T04:16:25.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.076+0000 7fd11fe00640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd10c0a2e90 0x7fd10c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:25.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.076+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fd1200cfd60 con 0x7fd1281076f0 2024-08-01T04:16:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.077+0000 7fd127400640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd10c0a2e90 0x7fd10c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.077+0000 7fd127400640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd10c0a2e90 0x7fd10c0a5350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd1180099a0 tx=0x7fd118009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.079+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd120053df0 con 0x7fd1281076f0 2024-08-01T04:16:25.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.312+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd12802d5e0 con 0x7fd1281076f0 2024-08-01T04:16:25.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.314+0000 7fd11fe00640 1 -- 172.21.15.3:0/3850596394 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7fd12009c690 con 0x7fd1281076f0 2024-08-01T04:16:25.317 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:25.317 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:16:25.317 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:16:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd10c0a2e90 msgr2=0x7fd10c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd10c0a2e90 0x7fd10c0a5350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd1180099a0 tx=0x7fd118009500 comp rx=0 tx=0).stop 2024-08-01T04:16:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 msgr2=0x7fd128074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 0x7fd128074ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e970 tx=0x7fd12000ed70 comp rx=0 tx=0).stop 2024-08-01T04:16:25.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 shutdown_connections 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1281076f0 0x7fd128074ca0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd128102090 0x7fd128074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd10c0a2e90 0x7fd10c0a5350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 --2- 172.21.15.3:0/3850596394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1281016f0 0x7fd128074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 >> 172.21.15.3:0/3850596394 conn(0x7fd1280fd470 msgr2=0x7fd1280fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.317+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 shutdown_connections 2024-08-01T04:16:25.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:25.318+0000 7fd12d200640 1 -- 172.21.15.3:0/3850596394 wait complete. 2024-08-01T04:16:25.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:25 smithi003 ceph-mon[32262]: from='client.25153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:25.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:25 smithi003 ceph-mon[32262]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-08-01T04:16:25.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3850596394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:25.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:25 smithi163 ceph-mon[37383]: from='client.25153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:25.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:25 smithi163 ceph-mon[37383]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-08-01T04:16:25.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3850596394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:25 smithi160 ceph-mon[37224]: from='client.25153 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:25 smithi160 ceph-mon[37224]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-08-01T04:16:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3850596394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:26.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:16:26.677 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:27.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 -- 172.21.15.3:0/1587459212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb181016f0 msgr2=0x7fcb18101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:27.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 --2- 172.21.15.3:0/1587459212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb181016f0 0x7fcb18101b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009930 tx=0x7fcb0002a420 comp rx=0 tx=0).stop 2024-08-01T04:16:27.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 -- 172.21.15.3:0/1587459212 shutdown_connections 2024-08-01T04:16:27.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 --2- 172.21.15.3:0/1587459212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb18102090 0x7fcb1810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 --2- 172.21.15.3:0/1587459212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb181016f0 0x7fcb18101b50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 --2- 172.21.15.3:0/1587459212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181076f0 0x7fcb18107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 -- 172.21.15.3:0/1587459212 >> 172.21.15.3:0/1587459212 conn(0x7fcb180fd470 msgr2=0x7fcb180ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:27.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 -- 172.21.15.3:0/1587459212 shutdown_connections 2024-08-01T04:16:27.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.380+0000 7fcb1d400640 1 -- 172.21.15.3:0/1587459212 wait complete. 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 Processor -- start 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 -- start start 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 0x7fcb1819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb18102090 0x7fcb1819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb181076f0 0x7fcb1819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18111b00 con 0x7fcb18102090 2024-08-01T04:16:27.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18111980 con 0x7fcb181076f0 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1d400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18111c80 con 0x7fcb181016f0 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 0x7fcb1819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.382+0000 7fcb1ca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb181076f0 0x7fcb1819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb17e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 0x7fcb1819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:36874/0 (socket says 172.21.15.3:36874) 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb17e00640 1 -- 172.21.15.3:0/2756160754 learned_addr learned my addr 172.21.15.3:0/2756160754 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb17400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb18102090 0x7fcb1819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb1ca00640 1 -- 172.21.15.3:0/2756160754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 msgr2=0x7fcb1819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb1ca00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 0x7fcb1819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb1ca00640 1 -- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb18102090 msgr2=0x7fcb1819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:27.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb1ca00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb18102090 0x7fcb1819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb1ca00640 1 -- 172.21.15.3:0/2756160754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1819f180 con 0x7fcb181076f0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb17e00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 0x7fcb1819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.383+0000 7fcb1ca00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb181076f0 0x7fcb1819eaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009fd0 tx=0x7fcb1000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.384+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000ce20 con 0x7fcb181076f0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.384+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb10005020 con 0x7fcb181076f0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.384+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1819f470 con 0x7fcb181076f0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.384+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10010720 con 0x7fcb181076f0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.384+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181a6d50 con 0x7fcb181076f0 2024-08-01T04:16:27.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.385+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb18101b50 con 0x7fcb181076f0 2024-08-01T04:16:27.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.387+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7fcb10005540 con 0x7fcb181076f0 2024-08-01T04:16:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.389+0000 7fcb14c00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcafc0a2e10 0x7fcafc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.389+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fcb100cfe80 con 0x7fcb181076f0 2024-08-01T04:16:27.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.389+0000 7fcb17e00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcafc0a2e10 0x7fcafc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.390+0000 7fcb17e00640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcafc0a2e10 0x7fcafc0a52d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb08011440 tx=0x7fcb08005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:27.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.390+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcb1009c7b0 con 0x7fcb181076f0 2024-08-01T04:16:27.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.587+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb1819b530 con 0x7fcb181076f0 2024-08-01T04:16:27.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.589+0000 7fcb14c00640 1 -- 172.21.15.3:0/2756160754 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23178 (secure 0 0 0) 0x7fcb1819b530 con 0x7fcb181076f0 2024-08-01T04:16:27.592 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:27.592 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:16:15.828445+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1348130449":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/346159818":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/4015366337":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2309230633":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:16:27.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.592+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcafc0a2e10 msgr2=0x7fcafc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:27.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.592+0000 7fcb1d400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcafc0a2e10 0x7fcafc0a52d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb08011440 tx=0x7fcb08005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:27.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.592+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb181076f0 msgr2=0x7fcb1819eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:27.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.592+0000 7fcb1d400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb181076f0 0x7fcb1819eaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009fd0 tx=0x7fcb1000ef10 comp rx=0 tx=0).stop 2024-08-01T04:16:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 shutdown_connections 2024-08-01T04:16:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb181076f0 0x7fcb1819eaa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb18102090 0x7fcb1819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fcafc0a2e10 0x7fcafc0a52d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 --2- 172.21.15.3:0/2756160754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb181016f0 0x7fcb1819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:27.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 >> 172.21.15.3:0/2756160754 conn(0x7fcb180fd470 msgr2=0x7fcb1810cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:27.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 shutdown_connections 2024-08-01T04:16:27.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:27.593+0000 7fcb1d400640 1 -- 172.21.15.3:0/2756160754 wait complete. 2024-08-01T04:16:27.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:27 smithi163 ceph-mon[37383]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 18 op/s 2024-08-01T04:16:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:27 smithi003 ceph-mon[32262]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 18 op/s 2024-08-01T04:16:27.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:27 smithi160 ceph-mon[37224]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 18 op/s 2024-08-01T04:16:28.468 DEBUG:tasks.fwd_scrub:Yielding 2024-08-01T04:16:28.469 INFO:teuthology.run_tasks:Running task exec... 2024-08-01T04:16:28.478 INFO:teuthology.task.exec:Executing custom commands... 2024-08-01T04:16:28.478 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:16:28.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-08-01T04:16:28.479 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-01T04:16:28.479 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:16:28.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:16:28.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:28 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2756160754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 -- 172.21.15.3:0/398972207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54103830 msgr2=0x7f6c54103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 --2- 172.21.15.3:0/398972207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54103830 0x7f6c54103c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009900 tx=0x7f6c4002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 -- 172.21.15.3:0/398972207 shutdown_connections 2024-08-01T04:16:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 --2- 172.21.15.3:0/398972207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 0x7f6c54110700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 --2- 172.21.15.3:0/398972207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54103830 0x7f6c54103c90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 --2- 172.21.15.3:0/398972207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54109840 0x7f6c54109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 -- 172.21.15.3:0/398972207 >> 172.21.15.3:0/398972207 conn(0x7f6c540ff4f0 msgr2=0x7f6c54101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 -- 172.21.15.3:0/398972207 shutdown_connections 2024-08-01T04:16:28.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.671+0000 7f6c53200640 1 -- 172.21.15.3:0/398972207 wait complete. 2024-08-01T04:16:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.672+0000 7f6c53200640 1 Processor -- start 2024-08-01T04:16:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.672+0000 7f6c53200640 1 -- start start 2024-08-01T04:16:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c53200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 0x7f6c5419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c53200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 0x7f6c5419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:28.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c53200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54109840 0x7f6c5419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c53200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54113cd0 con 0x7f6c54103830 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c53200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54113b50 con 0x7f6c54109840 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c53200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54113e50 con 0x7f6c541041d0 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 0x7f6c5419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c51e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 0x7f6c5419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43674/0 (socket says 172.21.15.3:43674) 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c51e00640 1 -- 172.21.15.3:0/3089421476 learned_addr learned my addr 172.21.15.3:0/3089421476 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c51400640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 0x7f6c5419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:28.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.673+0000 7f6c52800640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54109840 0x7f6c5419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51400640 1 -- 172.21.15.3:0/3089421476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54109840 msgr2=0x7f6c5419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51400640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54109840 0x7f6c5419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51400640 1 -- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 msgr2=0x7f6c5419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51400640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 0x7f6c5419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51400640 1 -- 172.21.15.3:0/3089421476 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5419f150 con 0x7f6c541041d0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c52800640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54109840 0x7f6c5419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51e00640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 0x7f6c5419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.674+0000 7f6c51400640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 0x7f6c5419a6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002a910 tx=0x7f6c40003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.675+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4003f070 con 0x7f6c541041d0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.675+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40031c20 con 0x7f6c541041d0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.675+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40035030 con 0x7f6c541041d0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.675+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5419f3e0 con 0x7f6c541041d0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.675+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541a6cd0 con 0x7f6c541041d0 2024-08-01T04:16:28.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.676+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c54103c90 con 0x7f6c541041d0 2024-08-01T04:16:28.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.678+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f6c40042070 con 0x7f6c541041d0 2024-08-01T04:16:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.679+0000 7f6c4aa00640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6c380a2e10 0x7f6c380a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.679+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6c400efd90 con 0x7f6c541041d0 2024-08-01T04:16:28.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.680+0000 7f6c51e00640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6c380a2e10 0x7f6c380a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:28.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.681+0000 7f6c51e00640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6c380a2e10 0x7f6c380a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e740 tx=0x7f6c44009800 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:28.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.684+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6c40031dd0 con 0x7f6c541041d0 2024-08-01T04:16:28.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:28 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2756160754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:28.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.895+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f6c541a6ed0 con 0x7f6c541041d0 2024-08-01T04:16:28.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:28 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2756160754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.903+0000 7f6c4aa00640 1 -- 172.21.15.3:0/3089421476 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f6c400bc140 con 0x7f6c541041d0 2024-08-01T04:16:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6c380a2e10 msgr2=0x7f6c380a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6c380a2e10 0x7f6c380a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400e740 tx=0x7f6c44009800 comp rx=0 tx=0).stop 2024-08-01T04:16:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 msgr2=0x7f6c5419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:28.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 0x7f6c5419a6a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c4002a910 tx=0x7f6c40003b90 comp rx=0 tx=0).stop 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 shutdown_connections 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c54109840 0x7f6c5419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6c380a2e10 0x7f6c380a52d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c541041d0 0x7f6c5419a6a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 --2- 172.21.15.3:0/3089421476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c54103830 0x7f6c5419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 >> 172.21.15.3:0/3089421476 conn(0x7f6c540ff4f0 msgr2=0x7f6c540ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 shutdown_connections 2024-08-01T04:16:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:28.906+0000 7f6c53200640 1 -- 172.21.15.3:0/3089421476 wait complete. 2024-08-01T04:16:28.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-08-01T04:16:28.967 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:29.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 -- 172.21.15.3:0/3906855780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c10dea0 msgr2=0x7f705c110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 --2- 172.21.15.3:0/3906855780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c10dea0 0x7f705c110290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ca0 tx=0x7f705402a940 comp rx=0 tx=0).stop 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 -- 172.21.15.3:0/3906855780 shutdown_connections 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 --2- 172.21.15.3:0/3906855780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c10dea0 0x7f705c110290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 --2- 172.21.15.3:0/3906855780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c074bf0 0x7f705c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 --2- 172.21.15.3:0/3906855780 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c0742d0 0x7f705c0746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 -- 172.21.15.3:0/3906855780 >> 172.21.15.3:0/3906855780 conn(0x7f705c0fd300 msgr2=0x7f705c0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 -- 172.21.15.3:0/3906855780 shutdown_connections 2024-08-01T04:16:29.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.155+0000 7f7061e00640 1 -- 172.21.15.3:0/3906855780 wait complete. 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.156+0000 7f7061e00640 1 Processor -- start 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.156+0000 7f7061e00640 1 -- start start 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 0x7f705c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c074bf0 0x7f705c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c10dea0 0x7f705c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f705c113c00 con 0x7f705c0742d0 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f705c113a80 con 0x7f705c074bf0 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f705c113d80 con 0x7f705c10dea0 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 0x7f705c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 0x7f705c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43686/0 (socket says 172.21.15.3:43686) 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 -- 172.21.15.3:0/319964124 learned_addr learned my addr 172.21.15.3:0/319964124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7061400640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c10dea0 0x7f705c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f705be00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c074bf0 0x7f705c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 -- 172.21.15.3:0/319964124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c10dea0 msgr2=0x7f705c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c10dea0 0x7f705c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 -- 172.21.15.3:0/319964124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c074bf0 msgr2=0x7f705c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c074bf0 0x7f705c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.157+0000 7f7060a00640 1 -- 172.21.15.3:0/319964124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c19f0d0 con 0x7f705c0742d0 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.158+0000 7f7060a00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 0x7f705c19a0e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f705000ea60 tx=0x7f705000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.158+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705000cde0 con 0x7f705c0742d0 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.158+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70500035f0 con 0x7f705c0742d0 2024-08-01T04:16:29.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.158+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c19f3c0 con 0x7f705c0742d0 2024-08-01T04:16:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.158+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050010670 con 0x7f705c0742d0 2024-08-01T04:16:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.158+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a6c50 con 0x7f705c0742d0 2024-08-01T04:16:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.159+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c0746b0 con 0x7f705c0742d0 2024-08-01T04:16:29.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.162+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f70500037a0 con 0x7f705c0742d0 2024-08-01T04:16:29.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.163+0000 7f7059600640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70440a2e10 0x7f70440a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.163+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f70500cfd00 con 0x7f705c0742d0 2024-08-01T04:16:29.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.164+0000 7f705be00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70440a2e10 0x7f70440a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.164+0000 7f705be00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70440a2e10 0x7f70440a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f705c10f2c0 tx=0x7f7048005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.170+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7050053df0 con 0x7f705c0742d0 2024-08-01T04:16:29.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.386+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f705c19b430 con 0x7f705c0742d0 2024-08-01T04:16:29.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.392+0000 7f7059600640 1 -- 172.21.15.3:0/319964124 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f705009c630 con 0x7f705c0742d0 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70440a2e10 msgr2=0x7f70440a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70440a2e10 0x7f70440a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f705c10f2c0 tx=0x7f7048005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 msgr2=0x7f705c19a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 0x7f705c19a0e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f705000ea60 tx=0x7f705000ef20 comp rx=0 tx=0).stop 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 shutdown_connections 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70440a2e10 0x7f70440a52d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c10dea0 0x7f705c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c074bf0 0x7f705c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 --2- 172.21.15.3:0/319964124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c0742d0 0x7f705c19a0e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 >> 172.21.15.3:0/319964124 conn(0x7f705c0fd300 msgr2=0x7f705c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 shutdown_connections 2024-08-01T04:16:29.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.396+0000 7f7061e00640 1 -- 172.21.15.3:0/319964124 wait complete. 2024-08-01T04:16:29.420 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-08-01T04:16:29.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 652 B/s wr, 17 op/s 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/319964124' entity='client.admin' 2024-08-01T04:16:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:29.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:29.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:29.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:29 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 -- 172.21.15.3:0/2801559510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 msgr2=0x7f9f2c10dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 --2- 172.21.15.3:0/2801559510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c10dba0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009900 tx=0x7f9f1802a480 comp rx=0 tx=0).stop 2024-08-01T04:16:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 -- 172.21.15.3:0/2801559510 shutdown_connections 2024-08-01T04:16:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 --2- 172.21.15.3:0/2801559510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c10e0e0 0x7f9f2c110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 --2- 172.21.15.3:0/2801559510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c10dba0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 --2- 172.21.15.3:0/2801559510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f2c074350 0x7f9f2c074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.588+0000 7f9f31a00640 1 -- 172.21.15.3:0/2801559510 >> 172.21.15.3:0/2801559510 conn(0x7f9f2c0fd5c0 msgr2=0x7f9f2c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.589+0000 7f9f31a00640 1 -- 172.21.15.3:0/2801559510 shutdown_connections 2024-08-01T04:16:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.589+0000 7f9f31a00640 1 -- 172.21.15.3:0/2801559510 wait complete. 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.590+0000 7f9f31a00640 1 Processor -- start 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.590+0000 7f9f31a00640 1 -- start start 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f31a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 0x7f9f2c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f31a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f31a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f2c10e0e0 0x7f9f2c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f31a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c113ee0 con 0x7f9f2c074c70 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f31a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c113d60 con 0x7f9f2c074350 2024-08-01T04:16:29.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f31a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c114060 con 0x7f9f2c10e0e0 2024-08-01T04:16:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 0x7f9f2c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 0x7f9f2c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33824/0 (socket says 172.21.15.3:33824) 2024-08-01T04:16:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f2aa00640 1 -- 172.21.15.3:0/3534878766 learned_addr learned my addr 172.21.15.3:0/3534878766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f2b400640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f2c10e0e0 0x7f9f2c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.591+0000 7f9f2a000640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2aa00640 1 -- 172.21.15.3:0/3534878766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f2c10e0e0 msgr2=0x7f9f2c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f2c10e0e0 0x7f9f2c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2aa00640 1 -- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 msgr2=0x7f9f2c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2aa00640 1 -- 172.21.15.3:0/3534878766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c19f360 con 0x7f9f2c074350 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2a000640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 0x7f9f2c19a3b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00e9f0 tx=0x7f9f1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f23400640 1 -- 172.21.15.3:0/3534878766 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00cd80 con 0x7f9f2c074350 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f23400640 1 -- 172.21.15.3:0/3534878766 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c010910 con 0x7f9f2c074350 2024-08-01T04:16:29.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f23400640 1 -- 172.21.15.3:0/3534878766 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c018990 con 0x7f9f2c074350 2024-08-01T04:16:29.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.592+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c19f650 con 0x7f9f2c074350 2024-08-01T04:16:29.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.593+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1a6f30 con 0x7f9f2c074350 2024-08-01T04:16:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.596+0000 7f9f23400640 1 -- 172.21.15.3:0/3534878766 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f9f1c010ac0 con 0x7f9f2c074350 2024-08-01T04:16:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.597+0000 7f9f344b4740 1 Processor -- start 2024-08-01T04:16:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 -- start start 2024-08-01T04:16:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007e1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007e1da0 0x5579007ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007ec660 0x5579007eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5579006fc7e0 con 0x5579007ec660 2024-08-01T04:16:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557900747ac0 con 0x5579007e1da0 2024-08-01T04:16:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.598+0000 7f9f344b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5579006df8c0 con 0x5579007e1480 2024-08-01T04:16:29.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.599+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007e1da0 0x5579007ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.599+0000 7f9f23400640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9f100a2e90 0x7f9f100a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.599+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007e1da0 0x5579007ec120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33826/0 (socket says 172.21.15.3:33826) 2024-08-01T04:16:29.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.599+0000 7f9f2aa00640 1 -- 172.21.15.3:0/1996850568 learned_addr learned my addr 172.21.15.3:0/1996850568 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.599+0000 7f9f2b400640 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007e1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.599+0000 7f9f23400640 1 -- 172.21.15.3:0/3534878766 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f9f1c0cfb00 con 0x7f9f2c074350 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 -- 172.21.15.3:0/1996850568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007e1da0 msgr2=0x5579007ec120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007e1da0 0x5579007ec120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2a000640 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007ec660 0x5579007eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 -- 172.21.15.3:0/1996850568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007ec660 msgr2=0x5579007eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007ec660 0x5579007eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 -- 172.21.15.3:0/1996850568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579006e6760 con 0x5579007e1480 2024-08-01T04:16:29.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2a000640 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007ec660 0x5579007eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:16:29.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9f100a2e90 0x7f9f100a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.600+0000 7f9f2b400640 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007e1860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9f240134b0 tx=0x7f9f24013970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3468c924c9bce server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.601+0000 7f9f17e00640 1 -- 172.21.15.3:0/1996850568 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24010430 con 0x5579007e1480 2024-08-01T04:16:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.601+0000 7f9f17e00640 1 -- 172.21.15.3:0/1996850568 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24010ab0 con 0x5579007e1480 2024-08-01T04:16:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.601+0000 7f9f17e00640 1 -- 172.21.15.3:0/1996850568 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2401b8e0 con 0x5579007e1480 2024-08-01T04:16:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 -- 172.21.15.3:0/1996850568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 msgr2=0x5579007e1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f2b400640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9f100a2e90 0x7f9f100a5350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f24023000 tx=0x7f9f24020000 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007e1860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9f240134b0 tx=0x7f9f24013970 comp rx=0 tx=0).stop 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 -- 172.21.15.3:0/1996850568 shutdown_connections 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007ec660 0x5579007eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007e1da0 0x5579007ec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1996850568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007e1860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 -- 172.21.15.3:0/1996850568 >> 172.21.15.3:0/1996850568 conn(0x5579007dda10 msgr2=0x5579007dddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.602+0000 7f9f344b4740 1 -- 172.21.15.3:0/1996850568 shutdown_connections 2024-08-01T04:16:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.603+0000 7f9f344b4740 1 -- 172.21.15.3:0/1996850568 wait complete. 2024-08-01T04:16:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.603+0000 7f9f344b4740 1 -- start start 2024-08-01T04:16:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.603+0000 7f9f344b4740 1 Processor -- start 2024-08-01T04:16:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007ea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557900702ae0 con 0x5579007e1480 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557900743a80 con 0x5579007ec660 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5579007ef090 con 0x5579007e1da0 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f344b4740 10 client.0 _ll_register_callbacks cb 0x557900704190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.604+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43722/0 (socket says 172.21.15.3:43722) 2024-08-01T04:16:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2aa00640 1 -- 172.21.15.3:0/639444315 learned_addr learned my addr 172.21.15.3:0/639444315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2b400640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007ea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2a000640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2b400640 1 -- 172.21.15.3:0/639444315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 msgr2=0x5579007e9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2b400640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007e9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2b400640 1 -- 172.21.15.3:0/639444315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 msgr2=0x5579007e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2b400640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.605+0000 7f9f2b400640 1 -- 172.21.15.3:0/639444315 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579004b96a0 con 0x5579007ec660 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.606+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.606+0000 7f9f2b400640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007ea180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9f24005f10 tx=0x7f9f24009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985dbdf4034f2c5d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.606+0000 7f9f14c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.606+0000 7f9f14c00640 1 -- 172.21.15.3:0/639444315 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24010430 con 0x5579007ec660 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.606+0000 7f9f14c00640 1 -- 172.21.15.3:0/639444315 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f9f240105d0 con 0x5579007ec660 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.606+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f24010ab0 con 0x5579007ec660 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.607+0000 7f9f14c00640 1 -- 172.21.15.3:0/639444315 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f24039050 con 0x5579007ec660 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.607+0000 7f9f344b4740 10 client.25214 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.607+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557900803510 con 0x5579007ec660 2024-08-01T04:16:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.607+0000 7f9f14c00640 1 -- 172.21.15.3:0/639444315 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24010ab0 con 0x5579007ec660 2024-08-01T04:16:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f14c00640 1 -- 172.21.15.3:0/639444315 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9f2401c850 con 0x5579007ec660 2024-08-01T04:16:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f344b4740 10 client.25214 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f344b4740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f344b4740 20 client.25214 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f344b4740 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557900804860 0x557900806c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f344b4740 4 client.25214 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.608+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557900755030 con 0x557900804860 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.609+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557900804860 0x557900806c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.609+0000 7f9f14c00640 10 client.25214 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:29.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.609+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557900804860 0x557900806c50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8b10075566d58c server_cookie=96a1e1e761672979 in_seq=0 out_seq=0 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f14c00640 1 -- 172.21.15.3:0/639444315 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557900755030 con 0x557900804860 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f14c00640 10 client.25214 handle_command_reply: tid=0 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 client.25214 shutdown 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 msgr2=0x5579007ea180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007ea180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9f24005f10 tx=0x7f9f24009d90 comp rx=0 tx=0).stop 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 shutdown_connections 2024-08-01T04:16:29.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557900804860 0x557900806c50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007ea180 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007e9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 --2- 172.21.15.3:0/639444315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 >> 172.21.15.3:0/639444315 conn(0x5579007dda10 msgr2=0x5579007e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.610+0000 7f9f14c00640 0 client.25214 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.611+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 shutdown_connections 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.611+0000 7f9f344b4740 1 -- 172.21.15.3:0/639444315 wait complete. 2024-08-01T04:16:29.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.611+0000 7f9f344b4740 20 client.25214 trim_cache size 0 max 16384 2024-08-01T04:16:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 -- 172.21.15.3:0/3586732396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9334102810 msgr2=0x7f9334102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 --2- 172.21.15.3:0/3586732396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9334102810 0x7f9334102bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9324009860 tx=0x7f932402a3d0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 -- 172.21.15.3:0/3586732396 shutdown_connections 2024-08-01T04:16:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 --2- 172.21.15.3:0/3586732396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f933410bc30 0x7f933410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 --2- 172.21.15.3:0/3586732396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9334103130 0x7f933410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 --2- 172.21.15.3:0/3586732396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9334102810 0x7f9334102bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 -- 172.21.15.3:0/3586732396 >> 172.21.15.3:0/3586732396 conn(0x7f93340774f0 msgr2=0x7f93340fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 -- 172.21.15.3:0/3586732396 shutdown_connections 2024-08-01T04:16:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.622+0000 7f9333e00640 1 -- 172.21.15.3:0/3586732396 wait complete. 2024-08-01T04:16:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.623+0000 7f9333e00640 1 Processor -- start 2024-08-01T04:16:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 -- start start 2024-08-01T04:16:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 0x7f9334197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9334103130 0x7f9334198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f933410bc30 0x7f933419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9334111a50 con 0x7f9334103130 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f93341118d0 con 0x7f9334102810 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9334111bd0 con 0x7f933410bc30 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9332000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9334103130 0x7f9334198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9332a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 0x7f9334197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9333400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f933410bc30 0x7f933419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9332a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 0x7f9334197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33846/0 (socket says 172.21.15.3:33846) 2024-08-01T04:16:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.624+0000 7f9332a00640 1 -- 172.21.15.3:0/2249829374 learned_addr learned my addr 172.21.15.3:0/2249829374 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.625+0000 7f9332a00640 1 -- 172.21.15.3:0/2249829374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f933410bc30 msgr2=0x7f933419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.625+0000 7f9332a00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f933410bc30 0x7f933419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.625+0000 7f9332a00640 1 -- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9334103130 msgr2=0x7f9334198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.625+0000 7f9332a00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9334103130 0x7f9334198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.625+0000 7f9332a00640 1 -- 172.21.15.3:0/2249829374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933419cf30 con 0x7f9334102810 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.625+0000 7f9332a00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 0x7f9334197f40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9324009980 tx=0x7f93240309a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.626+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324040070 con 0x7f9334102810 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.626+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9324005620 con 0x7f9334102810 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.626+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324004920 con 0x7f9334102810 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.626+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933419d1c0 con 0x7f9334102810 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.626+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9334100390 con 0x7f9334102810 2024-08-01T04:16:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.627+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9334102bf0 con 0x7f9334102810 2024-08-01T04:16:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.630+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f9324043070 con 0x7f9334102810 2024-08-01T04:16:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.632+0000 7f9338a00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f93180a2e90 0x7f93180a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.632+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f93240f0340 con 0x7f9334102810 2024-08-01T04:16:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.632+0000 7f9332000640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f93180a2e90 0x7f93180a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.633+0000 7f9332000640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f93180a2e90 0x7f93180a5350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f93340ffa70 tx=0x7f9320009260 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.636+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9324031400 con 0x7f9334102810 2024-08-01T04:16:29.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 Processor -- start 2024-08-01T04:16:29.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 -- start start 2024-08-01T04:16:29.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x55790080c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1adf10 con 0x5579007e1480 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1ab7e0 con 0x5579007ec660 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.638+0000 7f9f344b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1a9630 con 0x5579007e1da0 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43760/0 (socket says 172.21.15.3:43760) 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2aa00640 1 -- 172.21.15.3:0/322437537 learned_addr learned my addr 172.21.15.3:0/322437537 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x55790080c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 -- 172.21.15.3:0/322437537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 msgr2=0x5579007eb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 -- 172.21.15.3:0/322437537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 msgr2=0x5579007e9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 -- 172.21.15.3:0/322437537 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579007e8710 con 0x5579007ec660 2024-08-01T04:16:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.639+0000 7f9f2b400640 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x55790080c390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401b400 tx=0x7f9f24005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442ad6baa044cdee server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.640+0000 7f9f17400640 1 -- 172.21.15.3:0/322437537 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24010ab0 con 0x5579007ec660 2024-08-01T04:16:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.640+0000 7f9f17400640 1 -- 172.21.15.3:0/322437537 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24030070 con 0x5579007ec660 2024-08-01T04:16:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.640+0000 7f9f17400640 1 -- 172.21.15.3:0/322437537 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2401b7d0 con 0x5579007ec660 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 -- 172.21.15.3:0/322437537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 msgr2=0x55790080c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x55790080c390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401b400 tx=0x7f9f24005fd0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 -- 172.21.15.3:0/322437537 shutdown_connections 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x55790080c390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1da0 0x5579007eb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 --2- 172.21.15.3:0/322437537 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1480 0x5579007e9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 -- 172.21.15.3:0/322437537 >> 172.21.15.3:0/322437537 conn(0x5579007dda10 msgr2=0x5579007eaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 -- 172.21.15.3:0/322437537 shutdown_connections 2024-08-01T04:16:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 -- 172.21.15.3:0/322437537 wait complete. 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 -- start start 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.641+0000 7f9f344b4740 1 Processor -- start 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1da0 0x5579007e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1a9ca0 con 0x5579007e1da0 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1ab110 con 0x5579007ec660 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1aa370 con 0x5579007e1480 2024-08-01T04:16:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f344b4740 10 client.0 _ll_register_callbacks cb 0x7f9f1c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f2b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f2a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1da0 0x5579007e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f2b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33858/0 (socket says 172.21.15.3:33858) 2024-08-01T04:16:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.642+0000 7f9f2b400640 1 -- 172.21.15.3:0/1069021062 learned_addr learned my addr 172.21.15.3:0/1069021062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2a000640 1 -- 172.21.15.3:0/1069021062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 msgr2=0x5579007fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2a000640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2a000640 1 -- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 msgr2=0x5579007e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2a000640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2a000640 1 -- 172.21.15.3:0/1069021062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579007e7210 con 0x5579007e1da0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2b400640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f2a000640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1da0 0x5579007e64d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f1802aa70 tx=0x7f9f18005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfde919a9adbddee server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.643+0000 7f9f14c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f14c00640 1 -- 172.21.15.3:0/1069021062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18035040 con 0x5579007e1da0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f14c00640 1 -- 172.21.15.3:0/1069021062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f9f180046c0 con 0x5579007e1da0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f24030070 con 0x5579007e1da0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f14c00640 1 -- 172.21.15.3:0/1069021062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f18005d50 con 0x5579007e1da0 2024-08-01T04:16:29.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f344b4740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55790086f3a0 con 0x5579007e1da0 2024-08-01T04:16:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.644+0000 7f9f14c00640 1 -- 172.21.15.3:0/1069021062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24030070 con 0x5579007e1da0 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f14c00640 1 -- 172.21.15.3:0/1069021062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9f1804c050 con 0x5579007e1da0 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f344b4740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f344b4740 10 client.15081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f344b4740 20 client.15081 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55790086fb20 0x557900871f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f344b4740 4 client.15081 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55790074f540 con 0x55790086fb20 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.645+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55790086fb20 0x557900871f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.646+0000 7f9f2aa00640 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55790086fb20 0x557900871f40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4b2d9f5a168e94 server_cookie=83d0f47987832a60 in_seq=0 out_seq=0 2024-08-01T04:16:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.646+0000 7f9f14c00640 10 client.15081 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f14c00640 1 -- 172.21.15.3:0/1069021062 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55790074f540 con 0x55790086fb20 2024-08-01T04:16:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f14c00640 10 client.15081 handle_command_reply: tid=0 2024-08-01T04:16:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 client.15081 shutdown 2024-08-01T04:16:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1da0 msgr2=0x5579007e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1da0 0x5579007e64d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f1802aa70 tx=0x7f9f18005270 comp rx=0 tx=0).stop 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 shutdown_connections 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55790086fb20 0x557900871f40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5579007ec660 0x5579007e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5579007e1da0 0x5579007e64d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 --2- 172.21.15.3:0/1069021062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579007e1480 0x5579007fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f14c00640 0 client.15081 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 >> 172.21.15.3:0/1069021062 conn(0x5579007dda10 msgr2=0x5579007e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 shutdown_connections 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.647+0000 7f9f344b4740 1 -- 172.21.15.3:0/1069021062 wait complete. 2024-08-01T04:16:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.648+0000 7f9f344b4740 20 client.15081 trim_cache size 0 max 16384 2024-08-01T04:16:29.652 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:29.652 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:16:29.652 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "00bcbcea-e87b-4648-a5b1-861c54e76a49", 2024-08-01T04:16:29.652 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:16:29.652 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:29.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.648+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9f100a2e90 msgr2=0x7f9f100a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.648+0000 7f9f31a00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9f100a2e90 0x7f9f100a5350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f24023000 tx=0x7f9f24020000 comp rx=0 tx=0).stop 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.648+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 msgr2=0x7f9f2c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.648+0000 7f9f31a00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 0x7f9f2c19a3b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00e9f0 tx=0x7f9f1c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 shutdown_connections 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9f100a2e90 0x7f9f100a5350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f2c10e0e0 0x7f9f2c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f2c074c70 0x7f9f2c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 --2- 172.21.15.3:0/3534878766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f2c074350 0x7f9f2c19a3b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 >> 172.21.15.3:0/3534878766 conn(0x7f9f2c0fd5c0 msgr2=0x7f9f2c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 shutdown_connections 2024-08-01T04:16:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.649+0000 7f9f31a00640 1 -- 172.21.15.3:0/3534878766 wait complete. 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 652 B/s wr, 17 op/s 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/319964124' entity='client.admin' 2024-08-01T04:16:29.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:29.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:29.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:29.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:29 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:29.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.864+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9334106c90 con 0x7f9334102810 2024-08-01T04:16:29.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.874+0000 7f9338a00640 1 -- 172.21.15.3:0/2249829374 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f93240bc6f0 con 0x7f9334102810 2024-08-01T04:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f93180a2e90 msgr2=0x7f93180a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f93180a2e90 0x7f93180a5350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f93340ffa70 tx=0x7f9320009260 comp rx=0 tx=0).stop 2024-08-01T04:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 msgr2=0x7f9334197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 0x7f9334197f40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9324009980 tx=0x7f93240309a0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 shutdown_connections 2024-08-01T04:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f93180a2e90 0x7f93180a5350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f933410bc30 0x7f933419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.878+0000 7f9333e00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9334103130 0x7f9334198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.879+0000 7f9333e00640 1 --2- 172.21.15.3:0/2249829374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9334102810 0x7f9334197f40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.879+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 >> 172.21.15.3:0/2249829374 conn(0x7f93340774f0 msgr2=0x7f93340fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.879+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 shutdown_connections 2024-08-01T04:16:29.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:29.879+0000 7f9333e00640 1 -- 172.21.15.3:0/2249829374 wait complete. 2024-08-01T04:16:29.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 652 B/s wr, 17 op/s 2024-08-01T04:16:29.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/319964124' entity='client.admin' 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:29.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:29 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:29.904 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-01T04:16:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 -- 172.21.15.3:0/3820164970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7701003d0 msgr2=0x7ff7701007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 --2- 172.21.15.3:0/3820164970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7701003d0 0x7ff7701007b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff760009900 tx=0x7ff76002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 -- 172.21.15.3:0/3820164970 shutdown_connections 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 --2- 172.21.15.3:0/3820164970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77010df80 0x7ff770110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 --2- 172.21.15.3:0/3820164970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff770100d80 0x7ff77010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 --2- 172.21.15.3:0/3820164970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7701003d0 0x7ff7701007b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 -- 172.21.15.3:0/3820164970 >> 172.21.15.3:0/3820164970 conn(0x7ff7700774f0 msgr2=0x7ff7700fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 -- 172.21.15.3:0/3820164970 shutdown_connections 2024-08-01T04:16:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.087+0000 7ff775400640 1 -- 172.21.15.3:0/3820164970 wait complete. 2024-08-01T04:16:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.088+0000 7ff775400640 1 Processor -- start 2024-08-01T04:16:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.088+0000 7ff775400640 1 -- start start 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff775400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7701003d0 0x7ff77019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff775400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 0x7ff77019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff775400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77010df80 0x7ff77019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff775400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff770113c80 con 0x7ff77010df80 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff775400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff770113b00 con 0x7ff7701003d0 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff775400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff770113e00 con 0x7ff770100d80 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff76f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 0x7ff77019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff76f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 0x7ff77019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37006/0 (socket says 172.21.15.3:37006) 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff774a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77010df80 0x7ff77019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff76f400640 1 -- 172.21.15.3:0/3566862141 learned_addr learned my addr 172.21.15.3:0/3566862141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff76fe00640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7701003d0 0x7ff77019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.089+0000 7ff76f400640 1 -- 172.21.15.3:0/3566862141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7701003d0 msgr2=0x7ff77019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76f400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7701003d0 0x7ff77019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76f400640 1 -- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77010df80 msgr2=0x7ff77019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76f400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77010df80 0x7ff77019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76f400640 1 -- 172.21.15.3:0/3566862141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77019f1a0 con 0x7ff770100d80 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76f400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 0x7ff77019a6f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00d770 tx=0x7ff75c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c017070 con 0x7ff770100d80 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77019f490 con 0x7ff770100d80 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75c005500 con 0x7ff770100d80 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c004e00 con 0x7ff770100d80 2024-08-01T04:16:30.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.090+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7701a6d20 con 0x7ff770100d80 2024-08-01T04:16:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.091+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7701007b0 con 0x7ff770100d80 2024-08-01T04:16:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.096+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7ff75c004fa0 con 0x7ff770100d80 2024-08-01T04:16:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.098+0000 7ff76cc00640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7540a2e10 0x7ff7540a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.098+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff75c0ced50 con 0x7ff770100d80 2024-08-01T04:16:30.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.098+0000 7ff76fe00640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7540a2e10 0x7ff7540a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.098+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff75c0cf250 con 0x7ff770100d80 2024-08-01T04:16:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.099+0000 7ff76fe00640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7540a2e10 0x7ff7540a52d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff760000c00 tx=0x7ff760005b50 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:30.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.321+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff770062360 con 0x7ff7540a2e10 2024-08-01T04:16:30.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.445+0000 7ff76cc00640 1 -- 172.21.15.3:0/3566862141 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff770062360 con 0x7ff7540a2e10 2024-08-01T04:16:30.447 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path / 2024-08-01T04:16:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7540a2e10 msgr2=0x7ff7540a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7540a2e10 0x7ff7540a52d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff760000c00 tx=0x7ff760005b50 comp rx=0 tx=0).stop 2024-08-01T04:16:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 msgr2=0x7ff77019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 0x7ff77019a6f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00d770 tx=0x7ff75c00dc30 comp rx=0 tx=0).stop 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 shutdown_connections 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff77010df80 0x7ff77019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7540a2e10 0x7ff7540a52d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff770100d80 0x7ff77019a6f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 --2- 172.21.15.3:0/3566862141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7701003d0 0x7ff77019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 >> 172.21.15.3:0/3566862141 conn(0x7ff7700774f0 msgr2=0x7ff7700ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 shutdown_connections 2024-08-01T04:16:30.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.449+0000 7ff775400640 1 -- 172.21.15.3:0/3566862141 wait complete. 2024-08-01T04:16:30.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:16:30.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 -- 172.21.15.3:0/3691351294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c100d40 msgr2=0x7f885c10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 --2- 172.21.15.3:0/3691351294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c100d40 0x7f885c10da00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 -- 172.21.15.3:0/3691351294 shutdown_connections 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 --2- 172.21.15.3:0/3691351294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c10df40 0x7f885c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 --2- 172.21.15.3:0/3691351294 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c100d40 0x7f885c10da00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 --2- 172.21.15.3:0/3691351294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f885c100390 0x7f885c100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 -- 172.21.15.3:0/3691351294 >> 172.21.15.3:0/3691351294 conn(0x7f885c0774f0 msgr2=0x7f885c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:30.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 -- 172.21.15.3:0/3691351294 shutdown_connections 2024-08-01T04:16:30.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.690+0000 7f8860e00640 1 -- 172.21.15.3:0/3691351294 wait complete. 2024-08-01T04:16:30.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 Processor -- start 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 -- start start 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f885c100390 0x7f885c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c100d40 0x7f885c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 0x7f885c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f885c113c10 con 0x7f885c10df40 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f885c113a90 con 0x7f885c100390 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.691+0000 7f8860e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f885c113d90 con 0x7f885c100d40 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c100d40 0x7f885c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 0x7f885c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c100d40 0x7f885c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37022/0 (socket says 172.21.15.3:37022) 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 0x7f885c19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43804/0 (socket says 172.21.15.3:43804) 2024-08-01T04:16:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885aa00640 1 -- 172.21.15.3:0/3373926286 learned_addr learned my addr 172.21.15.3:0/3373926286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885b400640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f885c100390 0x7f885c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 -- 172.21.15.3:0/3373926286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c100d40 msgr2=0x7f885c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c100d40 0x7f885c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 -- 172.21.15.3:0/3373926286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f885c100390 msgr2=0x7f885c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f885c100390 0x7f885c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.692+0000 7f885be00640 1 -- 172.21.15.3:0/3373926286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c19f110 con 0x7f885c10df40 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.693+0000 7f885be00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 0x7f885c19e9e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8854005fd0 tx=0x7f885400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.693+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854004d20 con 0x7f885c10df40 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.693+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8854005020 con 0x7f885c10df40 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.693+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540116d0 con 0x7f885c10df40 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.693+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c19f400 con 0x7f885c10df40 2024-08-01T04:16:30.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.693+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a6c90 con 0x7f885c10df40 2024-08-01T04:16:30.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.694+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c100770 con 0x7f885c10df40 2024-08-01T04:16:30.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.696+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f8854005540 con 0x7f885c10df40 2024-08-01T04:16:30.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.698+0000 7f8853e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f88400a2e90 0x7f88400a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:30.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.698+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f88540cfae0 con 0x7f885c10df40 2024-08-01T04:16:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.698+0000 7f885b400640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f88400a2e90 0x7f88400a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.699+0000 7f885b400640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f88400a2e90 0x7f88400a5350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f884c009bd0 tx=0x7f884c005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.700+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8854016030 con 0x7f885c10df40 2024-08-01T04:16:30.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:30.939+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f885c062360 con 0x7f88400a2e90 2024-08-01T04:16:30.972 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:31.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.019+0000 7f8853e00640 1 -- 172.21.15.3:0/3373926286 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f885c062360 con 0x7f88400a2e90 2024-08-01T04:16:31.021 INFO:teuthology.orchestra.run.smithi003.stdout:Retention added to path / 2024-08-01T04:16:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f88400a2e90 msgr2=0x7f88400a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f88400a2e90 0x7f88400a5350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f884c009bd0 tx=0x7f884c005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 msgr2=0x7f885c19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 0x7f885c19e9e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8854005fd0 tx=0x7f885400edd0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 shutdown_connections 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f885c10df40 0x7f885c19e9e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f88400a2e90 0x7f88400a5350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f885c100d40 0x7f885c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 --2- 172.21.15.3:0/3373926286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f885c100390 0x7f885c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 >> 172.21.15.3:0/3373926286 conn(0x7f885c0774f0 msgr2=0x7f885c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.021+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 shutdown_connections 2024-08-01T04:16:31.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.022+0000 7f8860e00640 1 -- 172.21.15.3:0/3373926286 wait complete. 2024-08-01T04:16:31.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:16:31.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:16:31.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:16:31.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 597 B/s wr, 15 op/s 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:31.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:30 smithi003 ceph-mon[32262]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:31.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-01T04:16:31.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:16:31.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:16:31.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 597 B/s wr, 15 op/s 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:31.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:30 smithi160 ceph-mon[37224]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 -- 172.21.15.3:0/4168398375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23941041b0 msgr2=0x7f23941106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 --2- 172.21.15.3:0/4168398375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23941041b0 0x7f23941106e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f238c009cd0 tx=0x7f238c02a8f0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 -- 172.21.15.3:0/4168398375 shutdown_connections 2024-08-01T04:16:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 --2- 172.21.15.3:0/4168398375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23941041b0 0x7f23941106e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 --2- 172.21.15.3:0/4168398375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2394103810 0x7f2394103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 --2- 172.21.15.3:0/4168398375 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394109820 0x7f2394109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 -- 172.21.15.3:0/4168398375 >> 172.21.15.3:0/4168398375 conn(0x7f23940ff4f0 msgr2=0x7f2394101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 -- 172.21.15.3:0/4168398375 shutdown_connections 2024-08-01T04:16:31.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.211+0000 7f2393e00640 1 -- 172.21.15.3:0/4168398375 wait complete. 2024-08-01T04:16:31.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.212+0000 7f2393e00640 1 Processor -- start 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 -- start start 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394103810 0x7f239419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23941041b0 0x7f239419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2394109820 0x7f239419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2394113cb0 con 0x7f23941041b0 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2394113b30 con 0x7f2394103810 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2394113e30 con 0x7f2394109820 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2394109820 0x7f239419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2392000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23941041b0 0x7f239419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2392a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394103810 0x7f239419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2394109820 0x7f239419ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37036/0 (socket says 172.21.15.3:37036) 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.213+0000 7f2393400640 1 -- 172.21.15.3:0/4193508839 learned_addr learned my addr 172.21.15.3:0/4193508839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392a00640 1 -- 172.21.15.3:0/4193508839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2394109820 msgr2=0x7f239419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392a00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2394109820 0x7f239419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392a00640 1 -- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23941041b0 msgr2=0x7f239419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392a00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23941041b0 0x7f239419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392a00640 1 -- 172.21.15.3:0/4193508839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239419f150 con 0x7f2394103810 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392000640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23941041b0 0x7f239419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:31.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2392a00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394103810 0x7f239419a160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f238400e9f0 tx=0x7f238400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238400ccc0 con 0x7f2394103810 2024-08-01T04:16:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2384005020 con 0x7f2394103810 2024-08-01T04:16:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239419f440 con 0x7f2394103810 2024-08-01T04:16:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23840049e0 con 0x7f2394103810 2024-08-01T04:16:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.214+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a6cd0 con 0x7f2394103810 2024-08-01T04:16:31.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.216+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2394103c70 con 0x7f2394103810 2024-08-01T04:16:31.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.218+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f2384005560 con 0x7f2394103810 2024-08-01T04:16:31.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.219+0000 7f2398a00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23780a2e10 0x7f23780a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.220+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f23840cf630 con 0x7f2394103810 2024-08-01T04:16:31.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.220+0000 7f2392000640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23780a2e10 0x7f23780a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.220+0000 7f2392000640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23780a2e10 0x7f23780a52d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2394108e60 tx=0x7f2380005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.221+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f23840051d0 con 0x7f2394103810 2024-08-01T04:16:31.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:16:31.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 597 B/s wr, 15 op/s 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:16:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:30 smithi163 ceph-mon[37383]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:31.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.450+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2394062360 con 0x7f23780a2e10 2024-08-01T04:16:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.464+0000 7f2398a00640 1 -- 172.21.15.3:0/4193508839 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2394062360 con 0x7f23780a2e10 2024-08-01T04:16:31.466 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-01T00:00:00", "created": "2024-08-01T04:16:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23780a2e10 msgr2=0x7f23780a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23780a2e10 0x7f23780a52d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2394108e60 tx=0x7f2380005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394103810 msgr2=0x7f239419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394103810 0x7f239419a160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f238400e9f0 tx=0x7f238400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 shutdown_connections 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23780a2e10 0x7f23780a52d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2394109820 0x7f239419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23941041b0 0x7f239419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 --2- 172.21.15.3:0/4193508839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2394103810 0x7f239419a160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.467+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 >> 172.21.15.3:0/4193508839 conn(0x7f23940ff4f0 msgr2=0x7f2394100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.468+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 shutdown_connections 2024-08-01T04:16:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.468+0000 7f2393e00640 1 -- 172.21.15.3:0/4193508839 wait complete. 2024-08-01T04:16:31.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-01T04:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 -- 172.21.15.3:0/4138713498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b3010bea0 msgr2=0x7f9b3010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 --2- 172.21.15.3:0/4138713498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b3010bea0 0x7f9b3010e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009ca0 tx=0x7f9b2802a950 comp rx=0 tx=0).stop 2024-08-01T04:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 -- 172.21.15.3:0/4138713498 shutdown_connections 2024-08-01T04:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 --2- 172.21.15.3:0/4138713498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b3010bea0 0x7f9b3010e340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 --2- 172.21.15.3:0/4138713498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b30103390 0x7f9b3010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 --2- 172.21.15.3:0/4138713498 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b30102a70 0x7f9b30102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 -- 172.21.15.3:0/4138713498 >> 172.21.15.3:0/4138713498 conn(0x7f9b300fc4d0 msgr2=0x7f9b300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 -- 172.21.15.3:0/4138713498 shutdown_connections 2024-08-01T04:16:31.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.669+0000 7f9b37e00640 1 -- 172.21.15.3:0/4138713498 wait complete. 2024-08-01T04:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.671+0000 7f9b37e00640 1 Processor -- start 2024-08-01T04:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.671+0000 7f9b37e00640 1 -- start start 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b30102a70 0x7f9b301006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 0x7f9b30100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3010bea0 0x7f9b30101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b37e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30111cb0 con 0x7f9b3010bea0 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b37e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30111b30 con 0x7f9b30103390 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b37e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30111e30 con 0x7f9b30102a70 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b34a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 0x7f9b30100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b35400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b30102a70 0x7f9b301006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b34a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 0x7f9b30100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33924/0 (socket says 172.21.15.3:33924) 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b34a00640 1 -- 172.21.15.3:0/84906889 learned_addr learned my addr 172.21.15.3:0/84906889 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.672+0000 7f9b35e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3010bea0 0x7f9b30101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b34a00640 1 -- 172.21.15.3:0/84906889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b30102a70 msgr2=0x7f9b301006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b34a00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b30102a70 0x7f9b301006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b34a00640 1 -- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3010bea0 msgr2=0x7f9b30101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b34a00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3010bea0 0x7f9b30101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b34a00640 1 -- 172.21.15.3:0/84906889 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b301a65d0 con 0x7f9b30103390 2024-08-01T04:16:31.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 -- 172.21.15.3:0/1110032147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db410bdf0 msgr2=0x7f7db410e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 --2- 172.21.15.3:0/1110032147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db410bdf0 0x7f7db410e2b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009ca0 tx=0x7f7dac02a950 comp rx=0 tx=0).stop 2024-08-01T04:16:31.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b34a00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 0x7f9b30100c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e7a0 tx=0x7f9b1c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 -- 172.21.15.3:0/1110032147 shutdown_connections 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 --2- 172.21.15.3:0/1110032147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db410bdf0 0x7f7db410e2b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 --2- 172.21.15.3:0/1110032147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7db41032f0 0x7f7db410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 --2- 172.21.15.3:0/1110032147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7db41029d0 0x7f7db4102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 -- 172.21.15.3:0/1110032147 >> 172.21.15.3:0/1110032147 conn(0x7f7db40fc420 msgr2=0x7f7db40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 -- 172.21.15.3:0/1110032147 shutdown_connections 2024-08-01T04:16:31.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f7dbc000640 1 -- 172.21.15.3:0/1110032147 wait complete. 2024-08-01T04:16:31.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b2e000640 1 -- 172.21.15.3:0/84906889 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00ccc0 con 0x7f9b30103390 2024-08-01T04:16:31.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301a68c0 con 0x7f9b30103390 2024-08-01T04:16:31.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b2e000640 1 -- 172.21.15.3:0/84906889 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b1c009530 con 0x7f9b30103390 2024-08-01T04:16:31.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.673+0000 7f9b2e000640 1 -- 172.21.15.3:0/84906889 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c010430 con 0x7f9b30103390 2024-08-01T04:16:31.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.674+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b301a6e00 con 0x7f9b30103390 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 Processor -- start 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 -- start start 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 0x7f7db410b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7db41032f0 0x7f7db4108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7db410bdf0 0x7f7db4108890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4101d30 con 0x7f7db41032f0 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4101bb0 con 0x7f7db41029d0 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.675+0000 7f7dbc000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4101eb0 con 0x7f7db410bdf0 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 0x7f7db410b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db8c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7db41032f0 0x7f7db4108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 0x7f7db410b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33932/0 (socket says 172.21.15.3:33932) 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 -- 172.21.15.3:0/3322474072 learned_addr learned my addr 172.21.15.3:0/3322474072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7dba000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7db410bdf0 0x7f7db4108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 -- 172.21.15.3:0/3322474072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7db410bdf0 msgr2=0x7f7db4108890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7db410bdf0 0x7f7db4108890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 -- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7db41032f0 msgr2=0x7f7db4108350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7db41032f0 0x7f7db4108350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 -- 172.21.15.3:0/3322474072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db4108dd0 con 0x7f7db41029d0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.676+0000 7f7db9600640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 0x7f7db410b2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7da800ea90 tx=0x7f7da800ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.677+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da800ce40 con 0x7f7db41029d0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.677+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da80035f0 con 0x7f7db41029d0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.677+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8010690 con 0x7f7db41029d0 2024-08-01T04:16:31.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.677+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db41090c0 con 0x7f7db41029d0 2024-08-01T04:16:31.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.677+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db41ab0b0 con 0x7f7db41029d0 2024-08-01T04:16:31.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.677+0000 7f9b3a807740 1 Processor -- start 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 -- start start 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bfc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b233b54bf0 con 0x55b233beeb90 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b233aecbd0 con 0x55b233bf9d70 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b3a807740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b233b50bb0 con 0x55b233bef4b0 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b35400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b35400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37100/0 (socket says 172.21.15.3:37100) 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.678+0000 7f9b35400640 1 -- 172.21.15.3:0/2509233118 learned_addr learned my addr 172.21.15.3:0/2509233118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b35e00640 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b2e000640 1 -- 172.21.15.3:0/84906889 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f9b1c005020 con 0x7f9b30103390 2024-08-01T04:16:31.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db406a910 con 0x7f7db41029d0 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b35e00640 1 -- 172.21.15.3:0/2509233118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 msgr2=0x55b233bf9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b35e00640 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b35e00640 1 -- 172.21.15.3:0/2509233118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 msgr2=0x55b233bfc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b34a00640 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bfc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b35e00640 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bfc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b35e00640 1 -- 172.21.15.3:0/2509233118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b233af38a0 con 0x55b233beeb90 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.679+0000 7f9b34a00640 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bfc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.680+0000 7f9b35e00640 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beef70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b28005e30 tx=0x7f9b28005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c083b0f709ecbe server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.680+0000 7f9b22600640 1 -- 172.21.15.3:0/2509233118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28048070 con 0x55b233beeb90 2024-08-01T04:16:31.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.680+0000 7f9b22600640 1 -- 172.21.15.3:0/2509233118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b28003e50 con 0x55b233beeb90 2024-08-01T04:16:31.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.680+0000 7f9b22600640 1 -- 172.21.15.3:0/2509233118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28034040 con 0x55b233beeb90 2024-08-01T04:16:31.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b2e000640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9b180a2e90 0x7f9b180a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b2e000640 1 -- 172.21.15.3:0/84906889 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f9b1c0cf9e0 con 0x7f9b30103390 2024-08-01T04:16:31.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f7da80037a0 con 0x7f7db41029d0 2024-08-01T04:16:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b35400640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9b180a2e90 0x7f9b180a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 -- 172.21.15.3:0/2509233118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 msgr2=0x55b233beef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beef70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b28005e30 tx=0x7f9b28005f80 comp rx=0 tx=0).stop 2024-08-01T04:16:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 -- 172.21.15.3:0/2509233118 shutdown_connections 2024-08-01T04:16:31.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bfc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2509233118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beef70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.681+0000 7f9b3a807740 1 -- 172.21.15.3:0/2509233118 >> 172.21.15.3:0/2509233118 conn(0x55b233beb120 msgr2=0x55b233beb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.682+0000 7f9b3a807740 1 -- 172.21.15.3:0/2509233118 shutdown_connections 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.682+0000 7f9b3a807740 1 -- 172.21.15.3:0/2509233118 wait complete. 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.682+0000 7f9b35400640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9b180a2e90 0x7f9b180a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55b233bebf80 tx=0x7f9b24005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.682+0000 7f9b3a807740 1 -- start start 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 Processor -- start 2024-08-01T04:16:31.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f7db2000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d9c0a2f40 0x7f7d9c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f7da80cfa70 con 0x7f7db41029d0 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 0x55b233bf6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bef4b0 0x55b233bf7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b233c05150 con 0x55b233bef4b0 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b233b0fd40 con 0x55b233bf9d70 2024-08-01T04:16:31.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b233c052d0 con 0x55b233beeb90 2024-08-01T04:16:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f7db8c00640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d9c0a2f40 0x7f7d9c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.683+0000 7f9b3a807740 10 client.0 _ll_register_callbacks cb 0x55b233ad8b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b34a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bef4b0 0x55b233bf7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 0x55b233bf6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 0x55b233bf6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37108/0 (socket says 172.21.15.3:37108) 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 -- 172.21.15.3:0/2217888061 learned_addr learned my addr 172.21.15.3:0/2217888061 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35e00640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 -- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 msgr2=0x55b233bf7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 -- 172.21.15.3:0/2217888061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bef4b0 msgr2=0x55b233bf7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bef4b0 0x55b233bf7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b35400640 1 -- 172.21.15.3:0/2217888061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2338c63c0 con 0x55b233beeb90 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f9b34a00640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bef4b0 0x55b233bf7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.684+0000 7f7db8c00640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d9c0a2f40 0x7f7d9c0a5400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7db4109b10 tx=0x7f7da0005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b35400640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 0x55b233bf6d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b24013010 tx=0x7f9b24004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3c131cd1281c45 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b16a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b16a00640 1 -- 172.21.15.3:0/2217888061 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2402e060 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b28003e50 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b16a00640 1 -- 172.21.15.3:0/2217888061 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f9b24018030 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b16a00640 1 -- 172.21.15.3:0/2217888061 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b24029420 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b3a807740 10 client.25234 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b233c10ec0 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.685+0000 7f9b16a00640 1 -- 172.21.15.3:0/2217888061 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b28003e50 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b16a00640 1 -- 172.21.15.3:0/2217888061 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9b2400f560 con 0x55b233beeb90 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b3a807740 10 client.25234 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b3a807740 10 client.25234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b3a807740 20 client.25234 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c122d0 0x55b233c146c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b3a807740 4 client.25234 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b233b62160 con 0x55b233c122d0 2024-08-01T04:16:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.686+0000 7f9b34a00640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c122d0 0x55b233c146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.687+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7da809c400 con 0x7f7db41029d0 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.687+0000 7f9b34a00640 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c122d0 0x55b233c146c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6019422db42c1478 server_cookie=e958180a07c7d261 in_seq=0 out_seq=0 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.687+0000 7f9b16a00640 10 client.25234 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b16a00640 1 -- 172.21.15.3:0/2217888061 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b233b62160 con 0x55b233c122d0 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b16a00640 10 client.25234 handle_command_reply: tid=0 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 client.25234 shutdown 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 msgr2=0x55b233bf6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 0x55b233bf6d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b24013010 tx=0x7f9b24004730 comp rx=0 tx=0).stop 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 shutdown_connections 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c122d0 0x55b233c146c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bef4b0 0x55b233bf7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 --2- 172.21.15.3:0/2217888061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233beeb90 0x55b233bf6d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b16a00640 0 client.25234 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 >> 172.21.15.3:0/2217888061 conn(0x55b233beb120 msgr2=0x55b233bf0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 shutdown_connections 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.688+0000 7f9b3a807740 1 -- 172.21.15.3:0/2217888061 wait complete. 2024-08-01T04:16:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.689+0000 7f9b3a807740 20 client.25234 trim_cache size 0 max 16384 2024-08-01T04:16:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.707+0000 7f9b3a807740 1 Processor -- start 2024-08-01T04:16:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 -- start start 2024-08-01T04:16:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301add20 con 0x55b233beeb90 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301ab5f0 con 0x55b233bf9d70 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b3a807740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301a9440 con 0x55b233bef4b0 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.708+0000 7f9b35400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b34a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43888/0 (socket says 172.21.15.3:43888) 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 -- 172.21.15.3:0/1353254047 learned_addr learned my addr 172.21.15.3:0/1353254047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 -- 172.21.15.3:0/1353254047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 msgr2=0x55b233bf15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 -- 172.21.15.3:0/1353254047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 msgr2=0x55b233bf8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 -- 172.21.15.3:0/1353254047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b233bfc4d0 con 0x55b233beeb90 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35e00640 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b34a00640 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.709+0000 7f9b35400640 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beb500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b24033040 tx=0x7f9b24004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6667b920adf2cac server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.710+0000 7f9b21c00640 1 -- 172.21.15.3:0/1353254047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2401e7c0 con 0x55b233beeb90 2024-08-01T04:16:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.710+0000 7f9b21c00640 1 -- 172.21.15.3:0/1353254047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2400fba0 con 0x55b233beeb90 2024-08-01T04:16:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.710+0000 7f9b21c00640 1 -- 172.21.15.3:0/1353254047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240313f0 con 0x55b233beeb90 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 -- 172.21.15.3:0/1353254047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 msgr2=0x55b233beb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beb500 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b24033040 tx=0x7f9b24004640 comp rx=0 tx=0).stop 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 -- 172.21.15.3:0/1353254047 shutdown_connections 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233bf9d70 0x55b233bf8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 --2- 172.21.15.3:0/1353254047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233beeb90 0x55b233beb500 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 -- 172.21.15.3:0/1353254047 >> 172.21.15.3:0/1353254047 conn(0x55b233beb120 msgr2=0x55b233bf8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 -- 172.21.15.3:0/1353254047 shutdown_connections 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.711+0000 7f9b3a807740 1 -- 172.21.15.3:0/1353254047 wait complete. 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.712+0000 7f9b3a807740 1 -- start start 2024-08-01T04:16:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.712+0000 7f9b3a807740 1 Processor -- start 2024-08-01T04:16:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233beeb90 0x55b233bf3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bf9d70 0x55b233bf4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301a9ab0 con 0x55b233bf9d70 2024-08-01T04:16:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301aaf20 con 0x55b233beeb90 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301aa180 con 0x55b233bef4b0 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b3a807740 10 client.0 _ll_register_callbacks cb 0x7f9b24009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b34a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b35e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bf9d70 0x55b233bf4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b35400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233beeb90 0x55b233bf3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b34a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37118/0 (socket says 172.21.15.3:37118) 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b34a00640 1 -- 172.21.15.3:0/3568120763 learned_addr learned my addr 172.21.15.3:0/3568120763 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b35e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bf9d70 0x55b233bf4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43898/0 (socket says 172.21.15.3:43898) 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.713+0000 7f9b35400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233beeb90 0x55b233bf3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33962/0 (socket says 172.21.15.3:33962) 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b34a00640 1 -- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233beeb90 msgr2=0x55b233bf3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b34a00640 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233beeb90 0x55b233bf3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b34a00640 1 -- 172.21.15.3:0/3568120763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bf9d70 msgr2=0x55b233bf4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b34a00640 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bf9d70 0x55b233bf4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b34a00640 1 -- 172.21.15.3:0/3568120763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b233c7bec0 con 0x55b233bef4b0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b34a00640 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf4150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e770 tx=0x7f9b1c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93466eb99d0750bb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b16a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b16a00640 1 -- 172.21.15.3:0/3568120763 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c09bc10 con 0x55b233bef4b0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b16a00640 1 -- 172.21.15.3:0/3568120763 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f9b1c005310 con 0x55b233bef4b0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.714+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2400fba0 con 0x55b233bef4b0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b16a00640 1 -- 172.21.15.3:0/3568120763 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b1c09ffc0 con 0x55b233bef4b0 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 10 client.25240 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b233c7d0b0 con 0x55b233bef4b0 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b16a00640 1 -- 172.21.15.3:0/3568120763 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2400fba0 con 0x55b233bef4b0 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b16a00640 1 -- 172.21.15.3:0/3568120763 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9b1c0ba020 con 0x55b233bef4b0 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 10 client.25240 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 10 client.25240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 20 client.25240 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c7d830 0x55b233c7fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 4 client.25240 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.715+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b233b5c670 con 0x55b233c7d830 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.716+0000 7f9b35400640 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c7d830 0x55b233c7fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.716+0000 7f9b35400640 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c7d830 0x55b233c7fc50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4f52fc4467f381 server_cookie=6f3c3839782f6777 in_seq=0 out_seq=0 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.716+0000 7f9b16a00640 10 client.25240 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b16a00640 1 -- 172.21.15.3:0/3568120763 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b233b5c670 con 0x55b233c7d830 2024-08-01T04:16:31.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b16a00640 10 client.25240 handle_command_reply: tid=0 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 client.25240 shutdown 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 msgr2=0x55b233bf4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf4150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e770 tx=0x7f9b1c005f10 comp rx=0 tx=0).stop 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 shutdown_connections 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b233c7d830 0x55b233c7fc50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b233bf9d70 0x55b233bf4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b233bef4b0 0x55b233bf4150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 --2- 172.21.15.3:0/3568120763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b233beeb90 0x55b233bf3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 >> 172.21.15.3:0/3568120763 conn(0x55b233beb120 msgr2=0x55b233bea280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b16a00640 0 client.25240 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 shutdown_connections 2024-08-01T04:16:31.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.717+0000 7f9b3a807740 1 -- 172.21.15.3:0/3568120763 wait complete. 2024-08-01T04:16:31.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b3a807740 20 client.25240 trim_cache size 0 max 16384 2024-08-01T04:16:31.722 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:31.722 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:16:31.722 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:16:31.722 INFO:teuthology.orchestra.run.smithi003.stdout: "00bcbcea-e87b-4648-a5b1-861c54e76a49": { 2024-08-01T04:16:31.722 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "00bcbcea-e87b-4648-a5b1-861c54e76a49", 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9b180a2e90 msgr2=0x7f9b180a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9b180a2e90 0x7f9b180a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55b233bebf80 tx=0x7f9b24005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 msgr2=0x7f9b30100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 0x7f9b30100c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00e7a0 tx=0x7f9b1c00ec60 comp rx=0 tx=0).stop 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 shutdown_connections 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b3010bea0 0x7f9b30101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9b30103390 0x7f9b30100c10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f9b180a2e90 0x7f9b180a5350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 --2- 172.21.15.3:0/84906889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b30102a70 0x7f9b301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.718+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 >> 172.21.15.3:0/84906889 conn(0x7f9b300fc4d0 msgr2=0x7f9b300fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.719+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 shutdown_connections 2024-08-01T04:16:31.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.719+0000 7f9b37e00640 1 -- 172.21.15.3:0/84906889 wait complete. 2024-08-01T04:16:31.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.926+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7db410ddb0 con 0x7f7d9c0a2f40 2024-08-01T04:16:31.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.939+0000 7f7db2000640 1 -- 172.21.15.3:0/3322474072 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7db410ddb0 con 0x7f7d9c0a2f40 2024-08-01T04:16:31.941 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-08-01T04:16:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d9c0a2f40 msgr2=0x7f7d9c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d9c0a2f40 0x7f7d9c0a5400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7db4109b10 tx=0x7f7da0005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:31.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 msgr2=0x7f7db410b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 0x7f7db410b2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7da800ea90 tx=0x7f7da800ef50 comp rx=0 tx=0).stop 2024-08-01T04:16:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 shutdown_connections 2024-08-01T04:16:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7d9c0a2f40 0x7f7d9c0a5400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7db410bdf0 0x7f7db4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7db41032f0 0x7f7db4108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 --2- 172.21.15.3:0/3322474072 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7db41029d0 0x7f7db410b2b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 >> 172.21.15.3:0/3322474072 conn(0x7f7db40fc420 msgr2=0x7f7db410d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 shutdown_connections 2024-08-01T04:16:31.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:31.942+0000 7f7dbc000640 1 -- 172.21.15.3:0/3322474072 wait complete. 2024-08-01T04:16:31.960 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:31 smithi003 ceph-mon[32262]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:31.961 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:31 smithi003 ceph-mon[32262]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:31.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-01T04:16:32.045 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-01T04:16:32.056 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-01T04:16:32.163 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-01T04:16:32.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:16:32.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:31 smithi160 ceph-mon[37224]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:32.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:31 smithi160 ceph-mon[37224]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:32.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:31 smithi163 ceph-mon[37383]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:32.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:31 smithi163 ceph-mon[37383]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:32.700 INFO:tasks.cephfs.filesystem:scrub status for tag:00bcbcea-e87b-4648-a5b1-861c54e76a49 - {'path': '/', 'tag': '00bcbcea-e87b-4648-a5b1-861c54e76a49', 'options': 'recursive,force'} 2024-08-01T04:16:32.760 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:33.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:32 smithi160 ceph-mon[37224]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-08-01T04:16:33.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:32 smithi160 ceph-mon[37224]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:33.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:32 smithi003 ceph-mon[32262]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-08-01T04:16:33.178 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:32 smithi003 ceph-mon[32262]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:32 smithi163 ceph-mon[37383]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 KiB/s wr, 20 op/s 2024-08-01T04:16:33.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:32 smithi163 ceph-mon[37383]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:16:33.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 -- 172.21.15.3:0/2396467098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 msgr2=0x7f27dc10da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2396467098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc10da60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009900 tx=0x7f27c802a490 comp rx=0 tx=0).stop 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 -- 172.21.15.3:0/2396467098 shutdown_connections 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2396467098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 0x7f27dc110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2396467098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc10da60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2396467098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 0x7f27dc074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 -- 172.21.15.3:0/2396467098 >> 172.21.15.3:0/2396467098 conn(0x7f27dc0fd3e0 msgr2=0x7f27dc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 -- 172.21.15.3:0/2396467098 shutdown_connections 2024-08-01T04:16:33.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.447+0000 7f27e2e00640 1 -- 172.21.15.3:0/2396467098 wait complete. 2024-08-01T04:16:33.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 Processor -- start 2024-08-01T04:16:33.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 -- start start 2024-08-01T04:16:33.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 0x7f27dc19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 0x7f27dc19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc113cd0 con 0x7f27dc074240 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc113b50 con 0x7f27dc10dfa0 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.449+0000 7f27e2e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc113e50 con 0x7f27dc074b60 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e2400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 0x7f27dc19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 0x7f27dc19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e2400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 0x7f27dc19ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:37360/0 (socket says 172.21.15.3:37360) 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e2400640 1 -- 172.21.15.3:0/2963541263 learned_addr learned my addr 172.21.15.3:0/2963541263 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:33.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1000640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1000640 1 -- 172.21.15.3:0/2963541263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 msgr2=0x7f27dc19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1000640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 0x7f27dc19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1000640 1 -- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 msgr2=0x7f27dc19a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1000640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 0x7f27dc19a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1000640 1 -- 172.21.15.3:0/2963541263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc19f100 con 0x7f27dc074b60 2024-08-01T04:16:33.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.450+0000 7f27e1a00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 0x7f27dc19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.451+0000 7f27e1000640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc19a690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27c8005940 tx=0x7f27c8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.451+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c803f030 con 0x7f27dc074b60 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.451+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c8003ad0 con 0x7f27dc074b60 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.451+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8035070 con 0x7f27dc074b60 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.451+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc19f390 con 0x7f27dc074b60 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.451+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc1a6cd0 con 0x7f27dc074b60 2024-08-01T04:16:33.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.453+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc075980 con 0x7f27dc074b60 2024-08-01T04:16:33.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.455+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f27c8042030 con 0x7f27dc074b60 2024-08-01T04:16:33.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.456+0000 7f27da000640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f27c40a2e10 0x7f27c40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:33.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.457+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f27c80ef8e0 con 0x7f27dc074b60 2024-08-01T04:16:33.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.457+0000 7f27e1a00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f27c40a2e10 0x7f27c40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:33.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.458+0000 7f27e1a00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f27c40a2e10 0x7f27c40a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009fc0 tx=0x7f27d0005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:33.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.460+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f27c80bbc90 con 0x7f27dc074b60 2024-08-01T04:16:33.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.651+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27dc109190 con 0x7f27c40a2e10 2024-08-01T04:16:33.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.670+0000 7f27da000640 1 -- 172.21.15.3:0/2963541263 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258042 (secure 0 0 0) 0x7f27d0004030 con 0x7f27c40a2e10 2024-08-01T04:16:33.679 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:33.681 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:16:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.678+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f27c40a2e10 msgr2=0x7f27c40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.678+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f27c40a2e10 0x7f27c40a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27d0009fc0 tx=0x7f27d0005f90 comp rx=0 tx=0).stop 2024-08-01T04:16:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.678+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 msgr2=0x7f27dc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.678+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc19a690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27c8005940 tx=0x7f27c8004270 comp rx=0 tx=0).stop 2024-08-01T04:16:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 shutdown_connections 2024-08-01T04:16:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27dc10dfa0 0x7f27dc19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f27c40a2e10 0x7f27c40a52d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27dc074b60 0x7f27dc19a690 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 --2- 172.21.15.3:0/2963541263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f27dc074240 0x7f27dc19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 >> 172.21.15.3:0/2963541263 conn(0x7f27dc0fd3e0 msgr2=0x7f27dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 shutdown_connections 2024-08-01T04:16:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:33.679+0000 7f27e2e00640 1 -- 172.21.15.3:0/2963541263 wait complete. 2024-08-01T04:16:34.122 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:33 smithi003 ceph-mon[32262]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:34.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:33 smithi163 ceph-mon[37383]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:34.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:33 smithi160 ceph-mon[37224]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:34.683 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-08-01T04:16:31.868253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389845,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1225,"num_read_kb":945,"num_write":532,"num_write_kb":4854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":912,"ondisk_log_size":912,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365676,"kb_used_data":12336,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576148,"statfs":{"total":1151940427776,"available":1151565975552,"internally_reserved":0,"allocated":12632064,"data_stored":9694491,"data_compressed":366490,"data_compressed_allocated":491520,"data_compressed_original":3993600,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200322,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":270,"num_write":53,"num_write_kb":473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003856"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551165+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:16:16.551165+0000","last_peered":"2024-08-01T04:16:16.551165+0000","last_clean":"2024-08-01T04:16:16.551165+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:16:16.551165+0000","last_undegraded":"2024-08-01T04:16:16.551165+0000","last_fullsized":"2024-08-01T04:16:16.551165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212873+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:16:16.212873+0000","last_peered":"2024-08-01T04:16:16.212873+0000","last_clean":"2024-08-01T04:16:16.212873+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:16:16.212873+0000","last_undegraded":"2024-08-01T04:16:16.212873+0000","last_fullsized":"2024-08-01T04:16:16.212873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212612+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:16:16.212612+0000","last_peered":"2024-08-01T04:16:16.212612+0000","last_clean":"2024-08-01T04:16:16.212612+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:16:16.212612+0000","last_undegraded":"2024-08-01T04:16:16.212612+0000","last_fullsized":"2024-08-01T04:16:16.212612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840852+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:16:15.840852+0000","last_peered":"2024-08-01T04:16:15.840852+0000","last_clean":"2024-08-01T04:16:15.840852+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:16:15.840852+0000","last_undegraded":"2024-08-01T04:16:15.840852+0000","last_fullsized":"2024-08-01T04:16:15.840852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953698+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:16:15.953698+0000","last_peered":"2024-08-01T04:16:15.953698+0000","last_clean":"2024-08-01T04:16:15.953698+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:16:15.953698+0000","last_undegraded":"2024-08-01T04:16:15.953698+0000","last_fullsized":"2024-08-01T04:16:15.953698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967039+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:16:15.967039+0000","last_peered":"2024-08-01T04:16:15.967039+0000","last_clean":"2024-08-01T04:16:15.967039+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:16:15.967039+0000","last_undegraded":"2024-08-01T04:16:15.967039+0000","last_fullsized":"2024-08-01T04:16:15.967039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210138+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:16:16.210138+0000","last_peered":"2024-08-01T04:16:16.210138+0000","last_clean":"2024-08-01T04:16:16.210138+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:16:16.210138+0000","last_undegraded":"2024-08-01T04:16:16.210138+0000","last_fullsized":"2024-08-01T04:16:16.210138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879784+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:16:15.879784+0000","last_peered":"2024-08-01T04:16:15.879784+0000","last_clean":"2024-08-01T04:16:15.879784+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:16:15.879784+0000","last_undegraded":"2024-08-01T04:16:15.879784+0000","last_fullsized":"2024-08-01T04:16:15.879784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006355+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:16:16.006355+0000","last_peered":"2024-08-01T04:16:16.006355+0000","last_clean":"2024-08-01T04:16:16.006355+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:16:16.006355+0000","last_undegraded":"2024-08-01T04:16:16.006355+0000","last_fullsized":"2024-08-01T04:16:16.006355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967418+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:16:15.967418+0000","last_peered":"2024-08-01T04:16:15.967418+0000","last_clean":"2024-08-01T04:16:15.967418+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:16:15.967418+0000","last_undegraded":"2024-08-01T04:16:15.967418+0000","last_fullsized":"2024-08-01T04:16:15.967418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210488+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:16:16.210488+0000","last_peered":"2024-08-01T04:16:16.210488+0000","last_clean":"2024-08-01T04:16:16.210488+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:16:16.210488+0000","last_undegraded":"2024-08-01T04:16:16.210488+0000","last_fullsized":"2024-08-01T04:16:16.210488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841291+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:16:15.841291+0000","last_peered":"2024-08-01T04:16:15.841291+0000","last_clean":"2024-08-01T04:16:15.841291+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:16:15.841291+0000","last_undegraded":"2024-08-01T04:16:15.841291+0000","last_fullsized":"2024-08-01T04:16:15.841291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877759+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:16:15.877759+0000","last_peered":"2024-08-01T04:16:15.877759+0000","last_clean":"2024-08-01T04:16:15.877759+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:16:15.877759+0000","last_undegraded":"2024-08-01T04:16:15.877759+0000","last_fullsized":"2024-08-01T04:16:15.877759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211881+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:16:16.211881+0000","last_peered":"2024-08-01T04:16:16.211881+0000","last_clean":"2024-08-01T04:16:16.211881+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:16:16.211881+0000","last_undegraded":"2024-08-01T04:16:16.211881+0000","last_fullsized":"2024-08-01T04:16:16.211881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211595+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:16:16.211595+0000","last_peered":"2024-08-01T04:16:16.211595+0000","last_clean":"2024-08-01T04:16:16.211595+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:16:16.211595+0000","last_undegraded":"2024-08-01T04:16:16.211595+0000","last_fullsized":"2024-08-01T04:16:16.211595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005632+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:16:16.005632+0000","last_peered":"2024-08-01T04:16:16.005632+0000","last_clean":"2024-08-01T04:16:16.005632+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:16:16.005632+0000","last_undegraded":"2024-08-01T04:16:16.005632+0000","last_fullsized":"2024-08-01T04:16:16.005632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551271+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:16:16.551271+0000","last_peered":"2024-08-01T04:16:16.551271+0000","last_clean":"2024-08-01T04:16:16.551271+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:16:16.551271+0000","last_undegraded":"2024-08-01T04:16:16.551271+0000","last_fullsized":"2024-08-01T04:16:16.551271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840840+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:16:15.840840+0000","last_peered":"2024-08-01T04:16:15.840840+0000","last_clean":"2024-08-01T04:16:15.840840+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:16:15.840840+0000","last_undegraded":"2024-08-01T04:16:15.840840+0000","last_fullsized":"2024-08-01T04:16:15.840840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840916+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:16:15.840916+0000","last_peered":"2024-08-01T04:16:15.840916+0000","last_clean":"2024-08-01T04:16:15.840916+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:16:15.840916+0000","last_undegraded":"2024-08-01T04:16:15.840916+0000","last_fullsized":"2024-08-01T04:16:15.840916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550965+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:16:16.550965+0000","last_peered":"2024-08-01T04:16:16.550965+0000","last_clean":"2024-08-01T04:16:16.550965+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:16:16.550965+0000","last_undegraded":"2024-08-01T04:16:16.550965+0000","last_fullsized":"2024-08-01T04:16:16.550965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877859+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:16:15.877859+0000","last_peered":"2024-08-01T04:16:15.877859+0000","last_clean":"2024-08-01T04:16:15.877859+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:16:15.877859+0000","last_undegraded":"2024-08-01T04:16:15.877859+0000","last_fullsized":"2024-08-01T04:16:15.877859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006020+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:16:16.006020+0000","last_peered":"2024-08-01T04:16:16.006020+0000","last_clean":"2024-08-01T04:16:16.006020+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:16:16.006020+0000","last_undegraded":"2024-08-01T04:16:16.006020+0000","last_fullsized":"2024-08-01T04:16:16.006020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879660+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:16:15.879660+0000","last_peered":"2024-08-01T04:16:15.879660+0000","last_clean":"2024-08-01T04:16:15.879660+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:16:15.879660+0000","last_undegraded":"2024-08-01T04:16:15.879660+0000","last_fullsized":"2024-08-01T04:16:15.879660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953511+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:16:15.953511+0000","last_peered":"2024-08-01T04:16:15.953511+0000","last_clean":"2024-08-01T04:16:15.953511+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:16:15.953511+0000","last_undegraded":"2024-08-01T04:16:15.953511+0000","last_fullsized":"2024-08-01T04:16:15.953511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880212+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:16:15.880212+0000","last_peered":"2024-08-01T04:16:15.880212+0000","last_clean":"2024-08-01T04:16:15.880212+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:16:15.880212+0000","last_undegraded":"2024-08-01T04:16:15.880212+0000","last_fullsized":"2024-08-01T04:16:15.880212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840932+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:16:15.840932+0000","last_peered":"2024-08-01T04:16:15.840932+0000","last_clean":"2024-08-01T04:16:15.840932+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:16:15.840932+0000","last_undegraded":"2024-08-01T04:16:15.840932+0000","last_fullsized":"2024-08-01T04:16:15.840932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551039+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:16:16.551039+0000","last_peered":"2024-08-01T04:16:16.551039+0000","last_clean":"2024-08-01T04:16:16.551039+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:16:16.551039+0000","last_undegraded":"2024-08-01T04:16:16.551039+0000","last_fullsized":"2024-08-01T04:16:16.551039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877756+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:16:15.877756+0000","last_peered":"2024-08-01T04:16:15.877756+0000","last_clean":"2024-08-01T04:16:15.877756+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:16:15.877756+0000","last_undegraded":"2024-08-01T04:16:15.877756+0000","last_fullsized":"2024-08-01T04:16:15.877756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212112+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:16:16.212112+0000","last_peered":"2024-08-01T04:16:16.212112+0000","last_clean":"2024-08-01T04:16:16.212112+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:16:16.212112+0000","last_undegraded":"2024-08-01T04:16:16.212112+0000","last_fullsized":"2024-08-01T04:16:16.212112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840499+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:16:15.840499+0000","last_peered":"2024-08-01T04:16:15.840499+0000","last_clean":"2024-08-01T04:16:15.840499+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:16:15.840499+0000","last_undegraded":"2024-08-01T04:16:15.840499+0000","last_fullsized":"2024-08-01T04:16:15.840499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840592+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:16:15.840592+0000","last_peered":"2024-08-01T04:16:15.840592+0000","last_clean":"2024-08-01T04:16:15.840592+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:16:15.840592+0000","last_undegraded":"2024-08-01T04:16:15.840592+0000","last_fullsized":"2024-08-01T04:16:15.840592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550743+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:16:16.550743+0000","last_peered":"2024-08-01T04:16:16.550743+0000","last_clean":"2024-08-01T04:16:16.550743+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:16:16.550743+0000","last_undegraded":"2024-08-01T04:16:16.550743+0000","last_fullsized":"2024-08-01T04:16:16.550743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953699+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:16:15.953699+0000","last_peered":"2024-08-01T04:16:15.953699+0000","last_clean":"2024-08-01T04:16:15.953699+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:16:15.953699+0000","last_undegraded":"2024-08-01T04:16:15.953699+0000","last_fullsized":"2024-08-01T04:16:15.953699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":191,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:30.857842+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:16:30.857842+0000","last_peered":"2024-08-01T04:16:30.857842+0000","last_clean":"2024-08-01T04:16:30.857842+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:16:30.857842+0000","last_undegraded":"2024-08-01T04:16:30.857842+0000","last_fullsized":"2024-08-01T04:16:30.857842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963465+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:16:15.963465+0000","last_peered":"2024-08-01T04:16:15.963465+0000","last_clean":"2024-08-01T04:16:15.963465+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:16:15.963465+0000","last_undegraded":"2024-08-01T04:16:15.963465+0000","last_fullsized":"2024-08-01T04:16:15.963465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967109+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:16:15.967109+0000","last_peered":"2024-08-01T04:16:15.967109+0000","last_clean":"2024-08-01T04:16:15.967109+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:16:15.967109+0000","last_undegraded":"2024-08-01T04:16:15.967109+0000","last_fullsized":"2024-08-01T04:16:15.967109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877210+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:16:15.877210+0000","last_peered":"2024-08-01T04:16:15.877210+0000","last_clean":"2024-08-01T04:16:15.877210+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:16:15.877210+0000","last_undegraded":"2024-08-01T04:16:15.877210+0000","last_fullsized":"2024-08-01T04:16:15.877210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879869+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:16:15.879869+0000","last_peered":"2024-08-01T04:16:15.879869+0000","last_clean":"2024-08-01T04:16:15.879869+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:16:15.879869+0000","last_undegraded":"2024-08-01T04:16:15.879869+0000","last_fullsized":"2024-08-01T04:16:15.879869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963453+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:16:15.963453+0000","last_peered":"2024-08-01T04:16:15.963453+0000","last_clean":"2024-08-01T04:16:15.963453+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:16:15.963453+0000","last_undegraded":"2024-08-01T04:16:15.963453+0000","last_fullsized":"2024-08-01T04:16:15.963453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840696+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:16:15.840696+0000","last_peered":"2024-08-01T04:16:15.840696+0000","last_clean":"2024-08-01T04:16:15.840696+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:16:15.840696+0000","last_undegraded":"2024-08-01T04:16:15.840696+0000","last_fullsized":"2024-08-01T04:16:15.840696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":175,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:25.848771+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:16:25.848771+0000","last_peered":"2024-08-01T04:16:25.848771+0000","last_clean":"2024-08-01T04:16:25.848771+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:16:25.848771+0000","last_undegraded":"2024-08-01T04:16:25.848771+0000","last_fullsized":"2024-08-01T04:16:25.848771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006173+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:16:16.006173+0000","last_peered":"2024-08-01T04:16:16.006173+0000","last_clean":"2024-08-01T04:16:16.006173+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:16:16.006173+0000","last_undegraded":"2024-08-01T04:16:16.006173+0000","last_fullsized":"2024-08-01T04:16:16.006173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210517+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:16:16.210517+0000","last_peered":"2024-08-01T04:16:16.210517+0000","last_clean":"2024-08-01T04:16:16.210517+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:16:16.210517+0000","last_undegraded":"2024-08-01T04:16:16.210517+0000","last_fullsized":"2024-08-01T04:16:16.210517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213392+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:16:16.213392+0000","last_peered":"2024-08-01T04:16:16.213392+0000","last_clean":"2024-08-01T04:16:16.213392+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:16:16.213392+0000","last_undegraded":"2024-08-01T04:16:16.213392+0000","last_fullsized":"2024-08-01T04:16:16.213392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963763+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:16:15.963763+0000","last_peered":"2024-08-01T04:16:15.963763+0000","last_clean":"2024-08-01T04:16:15.963763+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:16:15.963763+0000","last_undegraded":"2024-08-01T04:16:15.963763+0000","last_fullsized":"2024-08-01T04:16:15.963763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006056+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:16:16.006056+0000","last_peered":"2024-08-01T04:16:16.006056+0000","last_clean":"2024-08-01T04:16:16.006056+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:16:16.006056+0000","last_undegraded":"2024-08-01T04:16:16.006056+0000","last_fullsized":"2024-08-01T04:16:16.006056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:26.850405+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:16:26.850405+0000","last_peered":"2024-08-01T04:16:26.850405+0000","last_clean":"2024-08-01T04:16:26.850405+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:16:26.850405+0000","last_undegraded":"2024-08-01T04:16:26.850405+0000","last_fullsized":"2024-08-01T04:16:26.850405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71563,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":288,"num_read_kb":213,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954043+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:16:15.954043+0000","last_peered":"2024-08-01T04:16:15.954043+0000","last_clean":"2024-08-01T04:16:15.954043+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:16:15.954043+0000","last_undegraded":"2024-08-01T04:16:15.954043+0000","last_fullsized":"2024-08-01T04:16:15.954043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840839+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:16:15.840839+0000","last_peered":"2024-08-01T04:16:15.840839+0000","last_clean":"2024-08-01T04:16:15.840839+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:16:15.840839+0000","last_undegraded":"2024-08-01T04:16:15.840839+0000","last_fullsized":"2024-08-01T04:16:15.840839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877540+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:16:15.877540+0000","last_peered":"2024-08-01T04:16:15.877540+0000","last_clean":"2024-08-01T04:16:15.877540+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:16:15.877540+0000","last_undegraded":"2024-08-01T04:16:15.877540+0000","last_fullsized":"2024-08-01T04:16:15.877540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963535+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:16:15.963535+0000","last_peered":"2024-08-01T04:16:15.963535+0000","last_clean":"2024-08-01T04:16:15.963535+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:16:15.963535+0000","last_undegraded":"2024-08-01T04:16:15.963535+0000","last_fullsized":"2024-08-01T04:16:15.963535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210232+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:16:16.210232+0000","last_peered":"2024-08-01T04:16:16.210232+0000","last_clean":"2024-08-01T04:16:16.210232+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:16:16.210232+0000","last_undegraded":"2024-08-01T04:16:16.210232+0000","last_fullsized":"2024-08-01T04:16:16.210232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":187,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:29.909140+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:16:29.909140+0000","last_peered":"2024-08-01T04:16:29.909140+0000","last_clean":"2024-08-01T04:16:29.909140+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:16:29.909140+0000","last_undegraded":"2024-08-01T04:16:29.909140+0000","last_fullsized":"2024-08-01T04:16:29.909140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210072+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:16:16.210072+0000","last_peered":"2024-08-01T04:16:16.210072+0000","last_clean":"2024-08-01T04:16:16.210072+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:16:16.210072+0000","last_undegraded":"2024-08-01T04:16:16.210072+0000","last_fullsized":"2024-08-01T04:16:16.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953840+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:16:15.953840+0000","last_peered":"2024-08-01T04:16:15.953840+0000","last_clean":"2024-08-01T04:16:15.953840+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:16:15.953840+0000","last_undegraded":"2024-08-01T04:16:15.953840+0000","last_fullsized":"2024-08-01T04:16:15.953840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963507+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:16:15.963507+0000","last_peered":"2024-08-01T04:16:15.963507+0000","last_clean":"2024-08-01T04:16:15.963507+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:16:15.963507+0000","last_undegraded":"2024-08-01T04:16:15.963507+0000","last_fullsized":"2024-08-01T04:16:15.963507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954076+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:16:15.954076+0000","last_peered":"2024-08-01T04:16:15.954076+0000","last_clean":"2024-08-01T04:16:15.954076+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:16:15.954076+0000","last_undegraded":"2024-08-01T04:16:15.954076+0000","last_fullsized":"2024-08-01T04:16:15.954076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551119+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:16:16.551119+0000","last_peered":"2024-08-01T04:16:16.551119+0000","last_clean":"2024-08-01T04:16:16.551119+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:16:16.551119+0000","last_undegraded":"2024-08-01T04:16:16.551119+0000","last_fullsized":"2024-08-01T04:16:16.551119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877775+0000","last_change":"2024-08-01T04:15:50.404577+0000","last_active":"2024-08-01T04:16:15.877775+0000","last_peered":"2024-08-01T04:16:15.877775+0000","last_clean":"2024-08-01T04:16:15.877775+0000","last_became_active":"2024-08-01T04:15:50.400018+0000","last_became_peered":"2024-08-01T04:15:50.400018+0000","last_unstale":"2024-08-01T04:16:15.877775+0000","last_undegraded":"2024-08-01T04:16:15.877775+0000","last_fullsized":"2024-08-01T04:16:15.877775+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:22:39.045462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210494+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:16:16.210494+0000","last_peered":"2024-08-01T04:16:16.210494+0000","last_clean":"2024-08-01T04:16:16.210494+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:16:16.210494+0000","last_undegraded":"2024-08-01T04:16:16.210494+0000","last_fullsized":"2024-08-01T04:16:16.210494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'65","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:31.016317+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:16:31.016317+0000","last_peered":"2024-08-01T04:16:31.016317+0000","last_clean":"2024-08-01T04:16:31.016317+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:16:31.016317+0000","last_undegraded":"2024-08-01T04:16:31.016317+0000","last_fullsized":"2024-08-01T04:16:31.016317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210202+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:16:16.210202+0000","last_peered":"2024-08-01T04:16:16.210202+0000","last_clean":"2024-08-01T04:16:16.210202+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:16:16.210202+0000","last_undegraded":"2024-08-01T04:16:16.210202+0000","last_fullsized":"2024-08-01T04:16:16.210202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841059+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:16:15.841059+0000","last_peered":"2024-08-01T04:16:15.841059+0000","last_clean":"2024-08-01T04:16:15.841059+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:16:15.841059+0000","last_undegraded":"2024-08-01T04:16:15.841059+0000","last_fullsized":"2024-08-01T04:16:15.841059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210445+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:16:16.210445+0000","last_peered":"2024-08-01T04:16:16.210445+0000","last_clean":"2024-08-01T04:16:16.210445+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:16:16.210445+0000","last_undegraded":"2024-08-01T04:16:16.210445+0000","last_fullsized":"2024-08-01T04:16:16.210445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:17.968134+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:16:17.968134+0000","last_peered":"2024-08-01T04:16:17.968134+0000","last_clean":"2024-08-01T04:16:17.968134+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:16:17.968134+0000","last_undegraded":"2024-08-01T04:16:17.968134+0000","last_fullsized":"2024-08-01T04:16:17.968134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841282+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:16:15.841282+0000","last_peered":"2024-08-01T04:16:15.841282+0000","last_clean":"2024-08-01T04:16:15.841282+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:16:15.841282+0000","last_undegraded":"2024-08-01T04:16:15.841282+0000","last_fullsized":"2024-08-01T04:16:15.841282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880005+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:16:15.880005+0000","last_peered":"2024-08-01T04:16:15.880005+0000","last_clean":"2024-08-01T04:16:15.880005+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:16:15.880005+0000","last_undegraded":"2024-08-01T04:16:15.880005+0000","last_fullsized":"2024-08-01T04:16:15.880005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210288+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:16:16.210288+0000","last_peered":"2024-08-01T04:16:16.210288+0000","last_clean":"2024-08-01T04:16:16.210288+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:16:16.210288+0000","last_undegraded":"2024-08-01T04:16:16.210288+0000","last_fullsized":"2024-08-01T04:16:16.210288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877149+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:16:15.877149+0000","last_peered":"2024-08-01T04:16:15.877149+0000","last_clean":"2024-08-01T04:16:15.877149+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:16:15.877149+0000","last_undegraded":"2024-08-01T04:16:15.877149+0000","last_fullsized":"2024-08-01T04:16:15.877149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":176,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:24.896996+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:16:24.896996+0000","last_peered":"2024-08-01T04:16:24.896996+0000","last_clean":"2024-08-01T04:16:24.896996+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:16:24.896996+0000","last_undegraded":"2024-08-01T04:16:24.896996+0000","last_fullsized":"2024-08-01T04:16:24.896996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963269+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:16:15.963269+0000","last_peered":"2024-08-01T04:16:15.963269+0000","last_clean":"2024-08-01T04:16:15.963269+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:16:15.963269+0000","last_undegraded":"2024-08-01T04:16:15.963269+0000","last_fullsized":"2024-08-01T04:16:15.963269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210461+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:16:16.210461+0000","last_peered":"2024-08-01T04:16:16.210461+0000","last_clean":"2024-08-01T04:16:16.210461+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:16:16.210461+0000","last_undegraded":"2024-08-01T04:16:16.210461+0000","last_fullsized":"2024-08-01T04:16:16.210461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967176+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:16:15.967176+0000","last_peered":"2024-08-01T04:16:15.967176+0000","last_clean":"2024-08-01T04:16:15.967176+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:16:15.967176+0000","last_undegraded":"2024-08-01T04:16:15.967176+0000","last_fullsized":"2024-08-01T04:16:15.967176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841074+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:16:15.841074+0000","last_peered":"2024-08-01T04:16:15.841074+0000","last_clean":"2024-08-01T04:16:15.841074+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:16:15.841074+0000","last_undegraded":"2024-08-01T04:16:15.841074+0000","last_fullsized":"2024-08-01T04:16:15.841074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967171+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:16:15.967171+0000","last_peered":"2024-08-01T04:16:15.967171+0000","last_clean":"2024-08-01T04:16:15.967171+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:16:15.967171+0000","last_undegraded":"2024-08-01T04:16:15.967171+0000","last_fullsized":"2024-08-01T04:16:15.967171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954025+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:16:15.954025+0000","last_peered":"2024-08-01T04:16:15.954025+0000","last_clean":"2024-08-01T04:16:15.954025+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:16:15.954025+0000","last_undegraded":"2024-08-01T04:16:15.954025+0000","last_fullsized":"2024-08-01T04:16:15.954025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877800+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:16:15.877800+0000","last_peered":"2024-08-01T04:16:15.877800+0000","last_clean":"2024-08-01T04:16:15.877800+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:16:15.877800+0000","last_undegraded":"2024-08-01T04:16:15.877800+0000","last_fullsized":"2024-08-01T04:16:15.877800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'49","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.545406+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:16:16.545406+0000","last_peered":"2024-08-01T04:16:16.545406+0000","last_clean":"2024-08-01T04:16:16.545406+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:16:16.545406+0000","last_undegraded":"2024-08-01T04:16:16.545406+0000","last_fullsized":"2024-08-01T04:16:16.545406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880558+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:16:15.880558+0000","last_peered":"2024-08-01T04:16:15.880558+0000","last_clean":"2024-08-01T04:16:15.880558+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:16:15.880558+0000","last_undegraded":"2024-08-01T04:16:15.880558+0000","last_fullsized":"2024-08-01T04:16:15.880558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841491+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:16:15.841491+0000","last_peered":"2024-08-01T04:16:15.841491+0000","last_clean":"2024-08-01T04:16:15.841491+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:16:15.841491+0000","last_undegraded":"2024-08-01T04:16:15.841491+0000","last_fullsized":"2024-08-01T04:16:15.841491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212996+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:16:16.212996+0000","last_peered":"2024-08-01T04:16:16.212996+0000","last_clean":"2024-08-01T04:16:16.212996+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:16:16.212996+0000","last_undegraded":"2024-08-01T04:16:16.212996+0000","last_fullsized":"2024-08-01T04:16:16.212996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005685+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:16:16.005685+0000","last_peered":"2024-08-01T04:16:16.005685+0000","last_clean":"2024-08-01T04:16:16.005685+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:16:16.005685+0000","last_undegraded":"2024-08-01T04:16:16.005685+0000","last_fullsized":"2024-08-01T04:16:16.005685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953515+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:16:15.953515+0000","last_peered":"2024-08-01T04:16:15.953515+0000","last_clean":"2024-08-01T04:16:15.953515+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:16:15.953515+0000","last_undegraded":"2024-08-01T04:16:15.953515+0000","last_fullsized":"2024-08-01T04:16:15.953515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879984+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:16:15.879984+0000","last_peered":"2024-08-01T04:16:15.879984+0000","last_clean":"2024-08-01T04:16:15.879984+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:16:15.879984+0000","last_undegraded":"2024-08-01T04:16:15.879984+0000","last_fullsized":"2024-08-01T04:16:15.879984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550822+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:16:16.550822+0000","last_peered":"2024-08-01T04:16:16.550822+0000","last_clean":"2024-08-01T04:16:16.550822+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:16:16.550822+0000","last_undegraded":"2024-08-01T04:16:16.550822+0000","last_fullsized":"2024-08-01T04:16:16.550822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550847+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:16:16.550847+0000","last_peered":"2024-08-01T04:16:16.550847+0000","last_clean":"2024-08-01T04:16:16.550847+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:16:16.550847+0000","last_undegraded":"2024-08-01T04:16:16.550847+0000","last_fullsized":"2024-08-01T04:16:16.550847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880233+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:16:15.880233+0000","last_peered":"2024-08-01T04:16:15.880233+0000","last_clean":"2024-08-01T04:16:15.880233+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:16:15.880233+0000","last_undegraded":"2024-08-01T04:16:15.880233+0000","last_fullsized":"2024-08-01T04:16:15.880233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953606+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:16:15.953606+0000","last_peered":"2024-08-01T04:16:15.953606+0000","last_clean":"2024-08-01T04:16:15.953606+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:16:15.953606+0000","last_undegraded":"2024-08-01T04:16:15.953606+0000","last_fullsized":"2024-08-01T04:16:15.953606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210241+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:16:16.210241+0000","last_peered":"2024-08-01T04:16:16.210241+0000","last_clean":"2024-08-01T04:16:16.210241+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:16:16.210241+0000","last_undegraded":"2024-08-01T04:16:16.210241+0000","last_fullsized":"2024-08-01T04:16:16.210241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963695+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:16:15.963695+0000","last_peered":"2024-08-01T04:16:15.963695+0000","last_clean":"2024-08-01T04:16:15.963695+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:16:15.963695+0000","last_undegraded":"2024-08-01T04:16:15.963695+0000","last_fullsized":"2024-08-01T04:16:15.963695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953907+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:16:15.953907+0000","last_peered":"2024-08-01T04:16:15.953907+0000","last_clean":"2024-08-01T04:16:15.953907+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:16:15.953907+0000","last_undegraded":"2024-08-01T04:16:15.953907+0000","last_fullsized":"2024-08-01T04:16:15.953907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005728+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:16:16.005728+0000","last_peered":"2024-08-01T04:16:16.005728+0000","last_clean":"2024-08-01T04:16:16.005728+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:16:16.005728+0000","last_undegraded":"2024-08-01T04:16:16.005728+0000","last_fullsized":"2024-08-01T04:16:16.005728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550806+0000","last_change":"2024-08-01T04:15:50.403853+0000","last_active":"2024-08-01T04:16:16.550806+0000","last_peered":"2024-08-01T04:16:16.550806+0000","last_clean":"2024-08-01T04:16:16.550806+0000","last_became_active":"2024-08-01T04:15:50.400282+0000","last_became_peered":"2024-08-01T04:15:50.400282+0000","last_unstale":"2024-08-01T04:16:16.550806+0000","last_undegraded":"2024-08-01T04:16:16.550806+0000","last_fullsized":"2024-08-01T04:16:16.550806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:11:52.649176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213209+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:16:16.213209+0000","last_peered":"2024-08-01T04:16:16.213209+0000","last_clean":"2024-08-01T04:16:16.213209+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:16:16.213209+0000","last_undegraded":"2024-08-01T04:16:16.213209+0000","last_fullsized":"2024-08-01T04:16:16.213209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880724+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:16:15.880724+0000","last_peered":"2024-08-01T04:16:15.880724+0000","last_clean":"2024-08-01T04:16:15.880724+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:16:15.880724+0000","last_undegraded":"2024-08-01T04:16:15.880724+0000","last_fullsized":"2024-08-01T04:16:15.880724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880592+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:16:15.880592+0000","last_peered":"2024-08-01T04:16:15.880592+0000","last_clean":"2024-08-01T04:16:15.880592+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:16:15.880592+0000","last_undegraded":"2024-08-01T04:16:15.880592+0000","last_fullsized":"2024-08-01T04:16:15.880592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840998+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:16:15.840998+0000","last_peered":"2024-08-01T04:16:15.840998+0000","last_clean":"2024-08-01T04:16:15.840998+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:16:15.840998+0000","last_undegraded":"2024-08-01T04:16:15.840998+0000","last_fullsized":"2024-08-01T04:16:15.840998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799243,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":988,"num_read_kb":615,"num_write":329,"num_write_kb":1914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":2004465,"data_compressed":84952,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449131,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":493767,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678065,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1289722,"data_compressed":56523,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841589,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":887219,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103228,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":887139,"data_compressed":33639,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1088084,"data_compressed":50367,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":493392,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":1414689,"data_compressed":65569,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":495140,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1018519,"data_compressed":39795,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":567409,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":564748,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":494663,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331019,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":206052,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:16:34.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:16:35.173 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:35.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:34 smithi163 ceph-mon[37383]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-08-01T04:16:35.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:34 smithi003 ceph-mon[32262]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-08-01T04:16:35.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:34 smithi160 ceph-mon[37224]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-08-01T04:16:35.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 -- 172.21.15.3:0/2262455121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074350 msgr2=0x7fa058074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:35.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 --2- 172.21.15.3:0/2262455121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074350 0x7fa058074730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa048009900 tx=0x7fa04802a480 comp rx=0 tx=0).stop 2024-08-01T04:16:35.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 -- 172.21.15.3:0/2262455121 shutdown_connections 2024-08-01T04:16:35.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 --2- 172.21.15.3:0/2262455121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa05810df40 0x7fa058110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:35.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 --2- 172.21.15.3:0/2262455121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074c70 0x7fa05810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:35.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 --2- 172.21.15.3:0/2262455121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074350 0x7fa058074730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:35.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 -- 172.21.15.3:0/2262455121 >> 172.21.15.3:0/2262455121 conn(0x7fa0580fd440 msgr2=0x7fa0580ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:35.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 -- 172.21.15.3:0/2262455121 shutdown_connections 2024-08-01T04:16:35.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.912+0000 7fa05ca00640 1 -- 172.21.15.3:0/2262455121 wait complete. 2024-08-01T04:16:35.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.913+0000 7fa05ca00640 1 Processor -- start 2024-08-01T04:16:35.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.913+0000 7fa05ca00640 1 -- start start 2024-08-01T04:16:35.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa05ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 0x7fa05819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:35.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa05ca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074c70 0x7fa05819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:35.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa05ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa05810df40 0x7fa05819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:35.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa05ca00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa058113d10 con 0x7fa058074350 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa05ca00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa058113b90 con 0x7fa058074c70 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa05ca00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa058113e90 con 0x7fa05810df40 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa056800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 0x7fa05819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa056800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 0x7fa05819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46044/0 (socket says 172.21.15.3:46044) 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa056800640 1 -- 172.21.15.3:0/260652888 learned_addr learned my addr 172.21.15.3:0/260652888 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa04de00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074c70 0x7fa05819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:35.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.914+0000 7fa057200640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa05810df40 0x7fa05819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04de00640 1 -- 172.21.15.3:0/260652888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa05810df40 msgr2=0x7fa05819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04de00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa05810df40 0x7fa05819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04de00640 1 -- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 msgr2=0x7fa05819a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04de00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 0x7fa05819a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04de00640 1 -- 172.21.15.3:0/260652888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05819f1c0 con 0x7fa058074c70 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa056800640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 0x7fa05819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04de00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074c70 0x7fa05819a750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e9f0 tx=0x7fa04000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:35.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04000ccc0 con 0x7fa058074c70 2024-08-01T04:16:35.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040010910 con 0x7fa058074c70 2024-08-01T04:16:35.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05819f4b0 con 0x7fa058074c70 2024-08-01T04:16:35.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.915+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040018960 con 0x7fa058074c70 2024-08-01T04:16:35.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.916+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a6d90 con 0x7fa058074c70 2024-08-01T04:16:35.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.917+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa058074730 con 0x7fa058074c70 2024-08-01T04:16:35.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.918+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7fa040010ac0 con 0x7fa058074c70 2024-08-01T04:16:35.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.920+0000 7fa04fe00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa03c0a2e90 0x7fa03c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.921+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa0400cfa90 con 0x7fa058074c70 2024-08-01T04:16:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.921+0000 7fa056800640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa03c0a2e90 0x7fa03c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:35.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.922+0000 7fa056800640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa03c0a2e90 0x7fa03c0a5350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0480096c0 tx=0x7fa0480046c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:35.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:35.924+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa04009bd70 con 0x7fa058074c70 2024-08-01T04:16:36.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.114+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa058062360 con 0x7fa03c0a2e90 2024-08-01T04:16:36.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.125+0000 7fa04fe00640 1 -- 172.21.15.3:0/260652888 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258056 (secure 0 0 0) 0x7fa058062360 con 0x7fa03c0a2e90 2024-08-01T04:16:36.128 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:36.131 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:16:36.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa03c0a2e90 msgr2=0x7fa03c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa03c0a2e90 0x7fa03c0a5350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0480096c0 tx=0x7fa0480046c0 comp rx=0 tx=0).stop 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074c70 msgr2=0x7fa05819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074c70 0x7fa05819a750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e9f0 tx=0x7fa04000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 shutdown_connections 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa03c0a2e90 0x7fa03c0a5350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa05810df40 0x7fa05819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa058074c70 0x7fa05819a750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 --2- 172.21.15.3:0/260652888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa058074350 0x7fa05819a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:36.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.130+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 >> 172.21.15.3:0/260652888 conn(0x7fa0580fd440 msgr2=0x7fa0580fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:36.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.131+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 shutdown_connections 2024-08-01T04:16:36.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:36.131+0000 7fa05ca00640 1 -- 172.21.15.3:0/260652888 wait complete. 2024-08-01T04:16:37.036 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-08-01T04:16:35.870115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389845,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1467,"num_read_kb":1160,"num_write":627,"num_write_kb":5302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369896,"kb_used_data":12384,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571928,"statfs":{"total":1151940427776,"available":1151561654272,"internally_reserved":0,"allocated":12681216,"data_stored":10087755,"data_compressed":385011,"data_compressed_allocated":516096,"data_compressed_original":4362240,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":252,"num_write":122,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004566"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551165+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:16:16.551165+0000","last_peered":"2024-08-01T04:16:16.551165+0000","last_clean":"2024-08-01T04:16:16.551165+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:16:16.551165+0000","last_undegraded":"2024-08-01T04:16:16.551165+0000","last_fullsized":"2024-08-01T04:16:16.551165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212873+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:16:16.212873+0000","last_peered":"2024-08-01T04:16:16.212873+0000","last_clean":"2024-08-01T04:16:16.212873+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:16:16.212873+0000","last_undegraded":"2024-08-01T04:16:16.212873+0000","last_fullsized":"2024-08-01T04:16:16.212873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212612+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:16:16.212612+0000","last_peered":"2024-08-01T04:16:16.212612+0000","last_clean":"2024-08-01T04:16:16.212612+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:16:16.212612+0000","last_undegraded":"2024-08-01T04:16:16.212612+0000","last_fullsized":"2024-08-01T04:16:16.212612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840852+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:16:15.840852+0000","last_peered":"2024-08-01T04:16:15.840852+0000","last_clean":"2024-08-01T04:16:15.840852+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:16:15.840852+0000","last_undegraded":"2024-08-01T04:16:15.840852+0000","last_fullsized":"2024-08-01T04:16:15.840852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953698+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:16:15.953698+0000","last_peered":"2024-08-01T04:16:15.953698+0000","last_clean":"2024-08-01T04:16:15.953698+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:16:15.953698+0000","last_undegraded":"2024-08-01T04:16:15.953698+0000","last_fullsized":"2024-08-01T04:16:15.953698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967039+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:16:15.967039+0000","last_peered":"2024-08-01T04:16:15.967039+0000","last_clean":"2024-08-01T04:16:15.967039+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:16:15.967039+0000","last_undegraded":"2024-08-01T04:16:15.967039+0000","last_fullsized":"2024-08-01T04:16:15.967039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210138+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:16:16.210138+0000","last_peered":"2024-08-01T04:16:16.210138+0000","last_clean":"2024-08-01T04:16:16.210138+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:16:16.210138+0000","last_undegraded":"2024-08-01T04:16:16.210138+0000","last_fullsized":"2024-08-01T04:16:16.210138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879784+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:16:15.879784+0000","last_peered":"2024-08-01T04:16:15.879784+0000","last_clean":"2024-08-01T04:16:15.879784+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:16:15.879784+0000","last_undegraded":"2024-08-01T04:16:15.879784+0000","last_fullsized":"2024-08-01T04:16:15.879784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006355+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:16:16.006355+0000","last_peered":"2024-08-01T04:16:16.006355+0000","last_clean":"2024-08-01T04:16:16.006355+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:16:16.006355+0000","last_undegraded":"2024-08-01T04:16:16.006355+0000","last_fullsized":"2024-08-01T04:16:16.006355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967418+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:16:15.967418+0000","last_peered":"2024-08-01T04:16:15.967418+0000","last_clean":"2024-08-01T04:16:15.967418+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:16:15.967418+0000","last_undegraded":"2024-08-01T04:16:15.967418+0000","last_fullsized":"2024-08-01T04:16:15.967418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210488+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:16:16.210488+0000","last_peered":"2024-08-01T04:16:16.210488+0000","last_clean":"2024-08-01T04:16:16.210488+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:16:16.210488+0000","last_undegraded":"2024-08-01T04:16:16.210488+0000","last_fullsized":"2024-08-01T04:16:16.210488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841291+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:16:15.841291+0000","last_peered":"2024-08-01T04:16:15.841291+0000","last_clean":"2024-08-01T04:16:15.841291+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:16:15.841291+0000","last_undegraded":"2024-08-01T04:16:15.841291+0000","last_fullsized":"2024-08-01T04:16:15.841291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877759+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:16:15.877759+0000","last_peered":"2024-08-01T04:16:15.877759+0000","last_clean":"2024-08-01T04:16:15.877759+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:16:15.877759+0000","last_undegraded":"2024-08-01T04:16:15.877759+0000","last_fullsized":"2024-08-01T04:16:15.877759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211881+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:16:16.211881+0000","last_peered":"2024-08-01T04:16:16.211881+0000","last_clean":"2024-08-01T04:16:16.211881+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:16:16.211881+0000","last_undegraded":"2024-08-01T04:16:16.211881+0000","last_fullsized":"2024-08-01T04:16:16.211881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211595+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:16:16.211595+0000","last_peered":"2024-08-01T04:16:16.211595+0000","last_clean":"2024-08-01T04:16:16.211595+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:16:16.211595+0000","last_undegraded":"2024-08-01T04:16:16.211595+0000","last_fullsized":"2024-08-01T04:16:16.211595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005632+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:16:16.005632+0000","last_peered":"2024-08-01T04:16:16.005632+0000","last_clean":"2024-08-01T04:16:16.005632+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:16:16.005632+0000","last_undegraded":"2024-08-01T04:16:16.005632+0000","last_fullsized":"2024-08-01T04:16:16.005632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551271+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:16:16.551271+0000","last_peered":"2024-08-01T04:16:16.551271+0000","last_clean":"2024-08-01T04:16:16.551271+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:16:16.551271+0000","last_undegraded":"2024-08-01T04:16:16.551271+0000","last_fullsized":"2024-08-01T04:16:16.551271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840840+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:16:15.840840+0000","last_peered":"2024-08-01T04:16:15.840840+0000","last_clean":"2024-08-01T04:16:15.840840+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:16:15.840840+0000","last_undegraded":"2024-08-01T04:16:15.840840+0000","last_fullsized":"2024-08-01T04:16:15.840840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840916+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:16:15.840916+0000","last_peered":"2024-08-01T04:16:15.840916+0000","last_clean":"2024-08-01T04:16:15.840916+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:16:15.840916+0000","last_undegraded":"2024-08-01T04:16:15.840916+0000","last_fullsized":"2024-08-01T04:16:15.840916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550965+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:16:16.550965+0000","last_peered":"2024-08-01T04:16:16.550965+0000","last_clean":"2024-08-01T04:16:16.550965+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:16:16.550965+0000","last_undegraded":"2024-08-01T04:16:16.550965+0000","last_fullsized":"2024-08-01T04:16:16.550965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877859+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:16:15.877859+0000","last_peered":"2024-08-01T04:16:15.877859+0000","last_clean":"2024-08-01T04:16:15.877859+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:16:15.877859+0000","last_undegraded":"2024-08-01T04:16:15.877859+0000","last_fullsized":"2024-08-01T04:16:15.877859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006020+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:16:16.006020+0000","last_peered":"2024-08-01T04:16:16.006020+0000","last_clean":"2024-08-01T04:16:16.006020+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:16:16.006020+0000","last_undegraded":"2024-08-01T04:16:16.006020+0000","last_fullsized":"2024-08-01T04:16:16.006020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879660+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:16:15.879660+0000","last_peered":"2024-08-01T04:16:15.879660+0000","last_clean":"2024-08-01T04:16:15.879660+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:16:15.879660+0000","last_undegraded":"2024-08-01T04:16:15.879660+0000","last_fullsized":"2024-08-01T04:16:15.879660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953511+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:16:15.953511+0000","last_peered":"2024-08-01T04:16:15.953511+0000","last_clean":"2024-08-01T04:16:15.953511+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:16:15.953511+0000","last_undegraded":"2024-08-01T04:16:15.953511+0000","last_fullsized":"2024-08-01T04:16:15.953511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880212+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:16:15.880212+0000","last_peered":"2024-08-01T04:16:15.880212+0000","last_clean":"2024-08-01T04:16:15.880212+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:16:15.880212+0000","last_undegraded":"2024-08-01T04:16:15.880212+0000","last_fullsized":"2024-08-01T04:16:15.880212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840932+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:16:15.840932+0000","last_peered":"2024-08-01T04:16:15.840932+0000","last_clean":"2024-08-01T04:16:15.840932+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:16:15.840932+0000","last_undegraded":"2024-08-01T04:16:15.840932+0000","last_fullsized":"2024-08-01T04:16:15.840932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551039+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:16:16.551039+0000","last_peered":"2024-08-01T04:16:16.551039+0000","last_clean":"2024-08-01T04:16:16.551039+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:16:16.551039+0000","last_undegraded":"2024-08-01T04:16:16.551039+0000","last_fullsized":"2024-08-01T04:16:16.551039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877756+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:16:15.877756+0000","last_peered":"2024-08-01T04:16:15.877756+0000","last_clean":"2024-08-01T04:16:15.877756+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:16:15.877756+0000","last_undegraded":"2024-08-01T04:16:15.877756+0000","last_fullsized":"2024-08-01T04:16:15.877756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212112+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:16:16.212112+0000","last_peered":"2024-08-01T04:16:16.212112+0000","last_clean":"2024-08-01T04:16:16.212112+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:16:16.212112+0000","last_undegraded":"2024-08-01T04:16:16.212112+0000","last_fullsized":"2024-08-01T04:16:16.212112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840499+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:16:15.840499+0000","last_peered":"2024-08-01T04:16:15.840499+0000","last_clean":"2024-08-01T04:16:15.840499+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:16:15.840499+0000","last_undegraded":"2024-08-01T04:16:15.840499+0000","last_fullsized":"2024-08-01T04:16:15.840499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840592+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:16:15.840592+0000","last_peered":"2024-08-01T04:16:15.840592+0000","last_clean":"2024-08-01T04:16:15.840592+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:16:15.840592+0000","last_undegraded":"2024-08-01T04:16:15.840592+0000","last_fullsized":"2024-08-01T04:16:15.840592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550743+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:16:16.550743+0000","last_peered":"2024-08-01T04:16:16.550743+0000","last_clean":"2024-08-01T04:16:16.550743+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:16:16.550743+0000","last_undegraded":"2024-08-01T04:16:16.550743+0000","last_fullsized":"2024-08-01T04:16:16.550743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953699+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:16:15.953699+0000","last_peered":"2024-08-01T04:16:15.953699+0000","last_clean":"2024-08-01T04:16:15.953699+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:16:15.953699+0000","last_undegraded":"2024-08-01T04:16:15.953699+0000","last_fullsized":"2024-08-01T04:16:15.953699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":191,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:30.857842+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:16:30.857842+0000","last_peered":"2024-08-01T04:16:30.857842+0000","last_clean":"2024-08-01T04:16:30.857842+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:16:30.857842+0000","last_undegraded":"2024-08-01T04:16:30.857842+0000","last_fullsized":"2024-08-01T04:16:30.857842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963465+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:16:15.963465+0000","last_peered":"2024-08-01T04:16:15.963465+0000","last_clean":"2024-08-01T04:16:15.963465+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:16:15.963465+0000","last_undegraded":"2024-08-01T04:16:15.963465+0000","last_fullsized":"2024-08-01T04:16:15.963465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967109+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:16:15.967109+0000","last_peered":"2024-08-01T04:16:15.967109+0000","last_clean":"2024-08-01T04:16:15.967109+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:16:15.967109+0000","last_undegraded":"2024-08-01T04:16:15.967109+0000","last_fullsized":"2024-08-01T04:16:15.967109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877210+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:16:15.877210+0000","last_peered":"2024-08-01T04:16:15.877210+0000","last_clean":"2024-08-01T04:16:15.877210+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:16:15.877210+0000","last_undegraded":"2024-08-01T04:16:15.877210+0000","last_fullsized":"2024-08-01T04:16:15.877210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879869+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:16:15.879869+0000","last_peered":"2024-08-01T04:16:15.879869+0000","last_clean":"2024-08-01T04:16:15.879869+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:16:15.879869+0000","last_undegraded":"2024-08-01T04:16:15.879869+0000","last_fullsized":"2024-08-01T04:16:15.879869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963453+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:16:15.963453+0000","last_peered":"2024-08-01T04:16:15.963453+0000","last_clean":"2024-08-01T04:16:15.963453+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:16:15.963453+0000","last_undegraded":"2024-08-01T04:16:15.963453+0000","last_fullsized":"2024-08-01T04:16:15.963453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840696+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:16:15.840696+0000","last_peered":"2024-08-01T04:16:15.840696+0000","last_clean":"2024-08-01T04:16:15.840696+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:16:15.840696+0000","last_undegraded":"2024-08-01T04:16:15.840696+0000","last_fullsized":"2024-08-01T04:16:15.840696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":180,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:30.857114+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:16:30.857114+0000","last_peered":"2024-08-01T04:16:30.857114+0000","last_clean":"2024-08-01T04:16:30.857114+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:16:30.857114+0000","last_undegraded":"2024-08-01T04:16:30.857114+0000","last_fullsized":"2024-08-01T04:16:30.857114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006173+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:16:16.006173+0000","last_peered":"2024-08-01T04:16:16.006173+0000","last_clean":"2024-08-01T04:16:16.006173+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:16:16.006173+0000","last_undegraded":"2024-08-01T04:16:16.006173+0000","last_fullsized":"2024-08-01T04:16:16.006173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210517+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:16:16.210517+0000","last_peered":"2024-08-01T04:16:16.210517+0000","last_clean":"2024-08-01T04:16:16.210517+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:16:16.210517+0000","last_undegraded":"2024-08-01T04:16:16.210517+0000","last_fullsized":"2024-08-01T04:16:16.210517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213392+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:16:16.213392+0000","last_peered":"2024-08-01T04:16:16.213392+0000","last_clean":"2024-08-01T04:16:16.213392+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:16:16.213392+0000","last_undegraded":"2024-08-01T04:16:16.213392+0000","last_fullsized":"2024-08-01T04:16:16.213392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963763+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:16:15.963763+0000","last_peered":"2024-08-01T04:16:15.963763+0000","last_clean":"2024-08-01T04:16:15.963763+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:16:15.963763+0000","last_undegraded":"2024-08-01T04:16:15.963763+0000","last_fullsized":"2024-08-01T04:16:15.963763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006056+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:16:16.006056+0000","last_peered":"2024-08-01T04:16:16.006056+0000","last_clean":"2024-08-01T04:16:16.006056+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:16:16.006056+0000","last_undegraded":"2024-08-01T04:16:16.006056+0000","last_fullsized":"2024-08-01T04:16:16.006056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":368,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:31.858848+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:16:31.858848+0000","last_peered":"2024-08-01T04:16:31.858848+0000","last_clean":"2024-08-01T04:16:31.858848+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:16:31.858848+0000","last_undegraded":"2024-08-01T04:16:31.858848+0000","last_fullsized":"2024-08-01T04:16:31.858848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71563,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":299,"num_read_kb":219,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954043+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:16:15.954043+0000","last_peered":"2024-08-01T04:16:15.954043+0000","last_clean":"2024-08-01T04:16:15.954043+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:16:15.954043+0000","last_undegraded":"2024-08-01T04:16:15.954043+0000","last_fullsized":"2024-08-01T04:16:15.954043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840839+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:16:15.840839+0000","last_peered":"2024-08-01T04:16:15.840839+0000","last_clean":"2024-08-01T04:16:15.840839+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:16:15.840839+0000","last_undegraded":"2024-08-01T04:16:15.840839+0000","last_fullsized":"2024-08-01T04:16:15.840839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877540+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:16:15.877540+0000","last_peered":"2024-08-01T04:16:15.877540+0000","last_clean":"2024-08-01T04:16:15.877540+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:16:15.877540+0000","last_undegraded":"2024-08-01T04:16:15.877540+0000","last_fullsized":"2024-08-01T04:16:15.877540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963535+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:16:15.963535+0000","last_peered":"2024-08-01T04:16:15.963535+0000","last_clean":"2024-08-01T04:16:15.963535+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:16:15.963535+0000","last_undegraded":"2024-08-01T04:16:15.963535+0000","last_fullsized":"2024-08-01T04:16:15.963535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210232+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:16:16.210232+0000","last_peered":"2024-08-01T04:16:16.210232+0000","last_clean":"2024-08-01T04:16:16.210232+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:16:16.210232+0000","last_undegraded":"2024-08-01T04:16:16.210232+0000","last_fullsized":"2024-08-01T04:16:16.210232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":187,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:29.909140+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:16:29.909140+0000","last_peered":"2024-08-01T04:16:29.909140+0000","last_clean":"2024-08-01T04:16:29.909140+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:16:29.909140+0000","last_undegraded":"2024-08-01T04:16:29.909140+0000","last_fullsized":"2024-08-01T04:16:29.909140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210072+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:16:16.210072+0000","last_peered":"2024-08-01T04:16:16.210072+0000","last_clean":"2024-08-01T04:16:16.210072+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:16:16.210072+0000","last_undegraded":"2024-08-01T04:16:16.210072+0000","last_fullsized":"2024-08-01T04:16:16.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953840+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:16:15.953840+0000","last_peered":"2024-08-01T04:16:15.953840+0000","last_clean":"2024-08-01T04:16:15.953840+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:16:15.953840+0000","last_undegraded":"2024-08-01T04:16:15.953840+0000","last_fullsized":"2024-08-01T04:16:15.953840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963507+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:16:15.963507+0000","last_peered":"2024-08-01T04:16:15.963507+0000","last_clean":"2024-08-01T04:16:15.963507+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:16:15.963507+0000","last_undegraded":"2024-08-01T04:16:15.963507+0000","last_fullsized":"2024-08-01T04:16:15.963507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954076+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:16:15.954076+0000","last_peered":"2024-08-01T04:16:15.954076+0000","last_clean":"2024-08-01T04:16:15.954076+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:16:15.954076+0000","last_undegraded":"2024-08-01T04:16:15.954076+0000","last_fullsized":"2024-08-01T04:16:15.954076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551119+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:16:16.551119+0000","last_peered":"2024-08-01T04:16:16.551119+0000","last_clean":"2024-08-01T04:16:16.551119+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:16:16.551119+0000","last_undegraded":"2024-08-01T04:16:16.551119+0000","last_fullsized":"2024-08-01T04:16:16.551119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877775+0000","last_change":"2024-08-01T04:15:50.404577+0000","last_active":"2024-08-01T04:16:15.877775+0000","last_peered":"2024-08-01T04:16:15.877775+0000","last_clean":"2024-08-01T04:16:15.877775+0000","last_became_active":"2024-08-01T04:15:50.400018+0000","last_became_peered":"2024-08-01T04:15:50.400018+0000","last_unstale":"2024-08-01T04:16:15.877775+0000","last_undegraded":"2024-08-01T04:16:15.877775+0000","last_fullsized":"2024-08-01T04:16:15.877775+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:22:39.045462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210494+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:16:16.210494+0000","last_peered":"2024-08-01T04:16:16.210494+0000","last_clean":"2024-08-01T04:16:16.210494+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:16:16.210494+0000","last_undegraded":"2024-08-01T04:16:16.210494+0000","last_fullsized":"2024-08-01T04:16:16.210494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'65","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:31.016317+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:16:31.016317+0000","last_peered":"2024-08-01T04:16:31.016317+0000","last_clean":"2024-08-01T04:16:31.016317+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:16:31.016317+0000","last_undegraded":"2024-08-01T04:16:31.016317+0000","last_fullsized":"2024-08-01T04:16:31.016317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210202+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:16:16.210202+0000","last_peered":"2024-08-01T04:16:16.210202+0000","last_clean":"2024-08-01T04:16:16.210202+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:16:16.210202+0000","last_undegraded":"2024-08-01T04:16:16.210202+0000","last_fullsized":"2024-08-01T04:16:16.210202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841059+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:16:15.841059+0000","last_peered":"2024-08-01T04:16:15.841059+0000","last_clean":"2024-08-01T04:16:15.841059+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:16:15.841059+0000","last_undegraded":"2024-08-01T04:16:15.841059+0000","last_fullsized":"2024-08-01T04:16:15.841059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210445+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:16:16.210445+0000","last_peered":"2024-08-01T04:16:16.210445+0000","last_clean":"2024-08-01T04:16:16.210445+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:16:16.210445+0000","last_undegraded":"2024-08-01T04:16:16.210445+0000","last_fullsized":"2024-08-01T04:16:16.210445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:17.968134+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:16:17.968134+0000","last_peered":"2024-08-01T04:16:17.968134+0000","last_clean":"2024-08-01T04:16:17.968134+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:16:17.968134+0000","last_undegraded":"2024-08-01T04:16:17.968134+0000","last_fullsized":"2024-08-01T04:16:17.968134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841282+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:16:15.841282+0000","last_peered":"2024-08-01T04:16:15.841282+0000","last_clean":"2024-08-01T04:16:15.841282+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:16:15.841282+0000","last_undegraded":"2024-08-01T04:16:15.841282+0000","last_fullsized":"2024-08-01T04:16:15.841282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880005+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:16:15.880005+0000","last_peered":"2024-08-01T04:16:15.880005+0000","last_clean":"2024-08-01T04:16:15.880005+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:16:15.880005+0000","last_undegraded":"2024-08-01T04:16:15.880005+0000","last_fullsized":"2024-08-01T04:16:15.880005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210288+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:16:16.210288+0000","last_peered":"2024-08-01T04:16:16.210288+0000","last_clean":"2024-08-01T04:16:16.210288+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:16:16.210288+0000","last_undegraded":"2024-08-01T04:16:16.210288+0000","last_fullsized":"2024-08-01T04:16:16.210288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877149+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:16:15.877149+0000","last_peered":"2024-08-01T04:16:15.877149+0000","last_clean":"2024-08-01T04:16:15.877149+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:16:15.877149+0000","last_undegraded":"2024-08-01T04:16:15.877149+0000","last_fullsized":"2024-08-01T04:16:15.877149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":181,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:29.908578+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:16:29.908578+0000","last_peered":"2024-08-01T04:16:29.908578+0000","last_clean":"2024-08-01T04:16:29.908578+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:16:29.908578+0000","last_undegraded":"2024-08-01T04:16:29.908578+0000","last_fullsized":"2024-08-01T04:16:29.908578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963269+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:16:15.963269+0000","last_peered":"2024-08-01T04:16:15.963269+0000","last_clean":"2024-08-01T04:16:15.963269+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:16:15.963269+0000","last_undegraded":"2024-08-01T04:16:15.963269+0000","last_fullsized":"2024-08-01T04:16:15.963269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210461+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:16:16.210461+0000","last_peered":"2024-08-01T04:16:16.210461+0000","last_clean":"2024-08-01T04:16:16.210461+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:16:16.210461+0000","last_undegraded":"2024-08-01T04:16:16.210461+0000","last_fullsized":"2024-08-01T04:16:16.210461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967176+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:16:15.967176+0000","last_peered":"2024-08-01T04:16:15.967176+0000","last_clean":"2024-08-01T04:16:15.967176+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:16:15.967176+0000","last_undegraded":"2024-08-01T04:16:15.967176+0000","last_fullsized":"2024-08-01T04:16:15.967176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841074+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:16:15.841074+0000","last_peered":"2024-08-01T04:16:15.841074+0000","last_clean":"2024-08-01T04:16:15.841074+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:16:15.841074+0000","last_undegraded":"2024-08-01T04:16:15.841074+0000","last_fullsized":"2024-08-01T04:16:15.841074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967171+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:16:15.967171+0000","last_peered":"2024-08-01T04:16:15.967171+0000","last_clean":"2024-08-01T04:16:15.967171+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:16:15.967171+0000","last_undegraded":"2024-08-01T04:16:15.967171+0000","last_fullsized":"2024-08-01T04:16:15.967171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954025+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:16:15.954025+0000","last_peered":"2024-08-01T04:16:15.954025+0000","last_clean":"2024-08-01T04:16:15.954025+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:16:15.954025+0000","last_undegraded":"2024-08-01T04:16:15.954025+0000","last_fullsized":"2024-08-01T04:16:15.954025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877800+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:16:15.877800+0000","last_peered":"2024-08-01T04:16:15.877800+0000","last_clean":"2024-08-01T04:16:15.877800+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:16:15.877800+0000","last_undegraded":"2024-08-01T04:16:15.877800+0000","last_fullsized":"2024-08-01T04:16:15.877800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'100","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:31.461604+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:16:31.461604+0000","last_peered":"2024-08-01T04:16:31.461604+0000","last_clean":"2024-08-01T04:16:31.461604+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:16:31.461604+0000","last_undegraded":"2024-08-01T04:16:31.461604+0000","last_fullsized":"2024-08-01T04:16:31.461604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":183,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880558+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:16:15.880558+0000","last_peered":"2024-08-01T04:16:15.880558+0000","last_clean":"2024-08-01T04:16:15.880558+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:16:15.880558+0000","last_undegraded":"2024-08-01T04:16:15.880558+0000","last_fullsized":"2024-08-01T04:16:15.880558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841491+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:16:15.841491+0000","last_peered":"2024-08-01T04:16:15.841491+0000","last_clean":"2024-08-01T04:16:15.841491+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:16:15.841491+0000","last_undegraded":"2024-08-01T04:16:15.841491+0000","last_fullsized":"2024-08-01T04:16:15.841491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212996+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:16:16.212996+0000","last_peered":"2024-08-01T04:16:16.212996+0000","last_clean":"2024-08-01T04:16:16.212996+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:16:16.212996+0000","last_undegraded":"2024-08-01T04:16:16.212996+0000","last_fullsized":"2024-08-01T04:16:16.212996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005685+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:16:16.005685+0000","last_peered":"2024-08-01T04:16:16.005685+0000","last_clean":"2024-08-01T04:16:16.005685+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:16:16.005685+0000","last_undegraded":"2024-08-01T04:16:16.005685+0000","last_fullsized":"2024-08-01T04:16:16.005685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953515+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:16:15.953515+0000","last_peered":"2024-08-01T04:16:15.953515+0000","last_clean":"2024-08-01T04:16:15.953515+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:16:15.953515+0000","last_undegraded":"2024-08-01T04:16:15.953515+0000","last_fullsized":"2024-08-01T04:16:15.953515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879984+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:16:15.879984+0000","last_peered":"2024-08-01T04:16:15.879984+0000","last_clean":"2024-08-01T04:16:15.879984+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:16:15.879984+0000","last_undegraded":"2024-08-01T04:16:15.879984+0000","last_fullsized":"2024-08-01T04:16:15.879984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550822+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:16:16.550822+0000","last_peered":"2024-08-01T04:16:16.550822+0000","last_clean":"2024-08-01T04:16:16.550822+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:16:16.550822+0000","last_undegraded":"2024-08-01T04:16:16.550822+0000","last_fullsized":"2024-08-01T04:16:16.550822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550847+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:16:16.550847+0000","last_peered":"2024-08-01T04:16:16.550847+0000","last_clean":"2024-08-01T04:16:16.550847+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:16:16.550847+0000","last_undegraded":"2024-08-01T04:16:16.550847+0000","last_fullsized":"2024-08-01T04:16:16.550847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880233+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:16:15.880233+0000","last_peered":"2024-08-01T04:16:15.880233+0000","last_clean":"2024-08-01T04:16:15.880233+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:16:15.880233+0000","last_undegraded":"2024-08-01T04:16:15.880233+0000","last_fullsized":"2024-08-01T04:16:15.880233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953606+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:16:15.953606+0000","last_peered":"2024-08-01T04:16:15.953606+0000","last_clean":"2024-08-01T04:16:15.953606+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:16:15.953606+0000","last_undegraded":"2024-08-01T04:16:15.953606+0000","last_fullsized":"2024-08-01T04:16:15.953606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210241+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:16:16.210241+0000","last_peered":"2024-08-01T04:16:16.210241+0000","last_clean":"2024-08-01T04:16:16.210241+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:16:16.210241+0000","last_undegraded":"2024-08-01T04:16:16.210241+0000","last_fullsized":"2024-08-01T04:16:16.210241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963695+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:16:15.963695+0000","last_peered":"2024-08-01T04:16:15.963695+0000","last_clean":"2024-08-01T04:16:15.963695+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:16:15.963695+0000","last_undegraded":"2024-08-01T04:16:15.963695+0000","last_fullsized":"2024-08-01T04:16:15.963695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953907+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:16:15.953907+0000","last_peered":"2024-08-01T04:16:15.953907+0000","last_clean":"2024-08-01T04:16:15.953907+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:16:15.953907+0000","last_undegraded":"2024-08-01T04:16:15.953907+0000","last_fullsized":"2024-08-01T04:16:15.953907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005728+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:16:16.005728+0000","last_peered":"2024-08-01T04:16:16.005728+0000","last_clean":"2024-08-01T04:16:16.005728+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:16:16.005728+0000","last_undegraded":"2024-08-01T04:16:16.005728+0000","last_fullsized":"2024-08-01T04:16:16.005728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550806+0000","last_change":"2024-08-01T04:15:50.403853+0000","last_active":"2024-08-01T04:16:16.550806+0000","last_peered":"2024-08-01T04:16:16.550806+0000","last_clean":"2024-08-01T04:16:16.550806+0000","last_became_active":"2024-08-01T04:15:50.400282+0000","last_became_peered":"2024-08-01T04:15:50.400282+0000","last_unstale":"2024-08-01T04:16:16.550806+0000","last_undegraded":"2024-08-01T04:16:16.550806+0000","last_fullsized":"2024-08-01T04:16:16.550806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:11:52.649176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213209+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:16:16.213209+0000","last_peered":"2024-08-01T04:16:16.213209+0000","last_clean":"2024-08-01T04:16:16.213209+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:16:16.213209+0000","last_undegraded":"2024-08-01T04:16:16.213209+0000","last_fullsized":"2024-08-01T04:16:16.213209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880724+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:16:15.880724+0000","last_peered":"2024-08-01T04:16:15.880724+0000","last_clean":"2024-08-01T04:16:15.880724+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:16:15.880724+0000","last_undegraded":"2024-08-01T04:16:15.880724+0000","last_fullsized":"2024-08-01T04:16:15.880724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880592+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:16:15.880592+0000","last_peered":"2024-08-01T04:16:15.880592+0000","last_clean":"2024-08-01T04:16:15.880592+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:16:15.880592+0000","last_undegraded":"2024-08-01T04:16:15.880592+0000","last_fullsized":"2024-08-01T04:16:15.880592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841053+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:16:15.841053+0000","last_peered":"2024-08-01T04:16:15.841053+0000","last_clean":"2024-08-01T04:16:15.841053+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:16:15.841053+0000","last_undegraded":"2024-08-01T04:16:15.841053+0000","last_fullsized":"2024-08-01T04:16:15.841053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799243,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1230,"num_read_kb":830,"num_write":424,"num_write_kb":2362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2397729,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":313,"ondisk_log_size":313,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449132,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":493767,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678066,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1200128,"data_stored":1486354,"data_compressed":65569,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":887219,"data_compressed":33782,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103229,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":887139,"data_compressed":33782,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1088084,"data_compressed":50367,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":493392,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757324,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":1414689,"data_compressed":65569,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920849,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":495140,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":1215151,"data_compressed":48984,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":567409,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":564748,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":494663,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331019,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402684,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:16:37.040 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-01T04:16:37.041 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph pg dump --format=json 2024-08-01T04:16:37.166 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:36 smithi003 ceph-mon[32262]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-08-01T04:16:37.166 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:36 smithi003 ceph-mon[32262]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:37.204 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:36 smithi160 ceph-mon[37224]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-08-01T04:16:37.204 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:36 smithi160 ceph-mon[37224]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:36 smithi163 ceph-mon[37383]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-08-01T04:16:37.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:36 smithi163 ceph-mon[37383]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:37.573 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.255+0000 7f08f4600640 1 -- 172.21.15.3:0/3883961727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108180 msgr2=0x7f08ec108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.255+0000 7f08f4600640 1 --2- 172.21.15.3:0/3883961727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108180 0x7f08ec108560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009ca0 tx=0x7f08e002a8b0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 -- 172.21.15.3:0/3883961727 shutdown_connections 2024-08-01T04:16:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 --2- 172.21.15.3:0/3883961727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 --2- 172.21.15.3:0/3883961727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec108b30 0x7f08ec10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 --2- 172.21.15.3:0/3883961727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108180 0x7f08ec108560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 -- 172.21.15.3:0/3883961727 >> 172.21.15.3:0/3883961727 conn(0x7f08ec0778d0 msgr2=0x7f08ec079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 -- 172.21.15.3:0/3883961727 shutdown_connections 2024-08-01T04:16:38.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.256+0000 7f08f4600640 1 -- 172.21.15.3:0/3883961727 wait complete. 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.257+0000 7f08f4600640 1 Processor -- start 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.257+0000 7f08f4600640 1 -- start start 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f4600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec108180 0x7f08ec19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f4600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 0x7f08ec19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f4600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec1a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f4600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec1182b0 con 0x7f08ec108180 2024-08-01T04:16:38.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f4600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec118130 con 0x7f08ec10d260 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f4600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec118430 con 0x7f08ec108b30 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 0x7f08ec19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f2600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec1a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec108180 0x7f08ec19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 0x7f08ec19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55340/0 (socket says 172.21.15.3:55340) 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f1200640 1 -- 172.21.15.3:0/3889853410 learned_addr learned my addr 172.21.15.3:0/3889853410 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:38.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.258+0000 7f08f2600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec1a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:37406/0 (socket says 172.21.15.3:37406) 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f2600640 1 -- 172.21.15.3:0/3889853410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 msgr2=0x7f08ec19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f2600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 0x7f08ec19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f2600640 1 -- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec108180 msgr2=0x7f08ec19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f2600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec108180 0x7f08ec19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f2600640 1 -- 172.21.15.3:0/3889853410 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec1a36e0 con 0x7f08ec10d260 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f1200640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 0x7f08ec19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08f2600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec1a3000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08e400e740 tx=0x7f08e400ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.259+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e400ccc0 con 0x7f08ec10d260 2024-08-01T04:16:38.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.260+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ec1a39d0 con 0x7f08ec10d260 2024-08-01T04:16:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.260+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e400ee50 con 0x7f08ec10d260 2024-08-01T04:16:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.260+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4004a00 con 0x7f08ec10d260 2024-08-01T04:16:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.260+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ec1ab2b0 con 0x7f08ec10d260 2024-08-01T04:16:38.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.261+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ec108560 con 0x7f08ec10d260 2024-08-01T04:16:38.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.263+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f08e4004ba0 con 0x7f08ec10d260 2024-08-01T04:16:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.264+0000 7f08ea000640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f08d40a2e10 0x7f08d40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:38.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.264+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f08e40cf610 con 0x7f08ec10d260 2024-08-01T04:16:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.265+0000 7f08f1c00640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f08d40a2e10 0x7f08d40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:38.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.265+0000 7f08f1c00640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f08d40a2e10 0x7f08d40a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006010 tx=0x7f08e0009820 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:38.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.272+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f08e409bf40 con 0x7f08ec10d260 2024-08-01T04:16:38.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.465+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08ec062360 con 0x7f08d40a2e10 2024-08-01T04:16:38.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.484+0000 7f08ea000640 1 -- 172.21.15.3:0/3889853410 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258056 (secure 0 0 0) 0x7f08ec062360 con 0x7f08d40a2e10 2024-08-01T04:16:38.488 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:38.492 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-08-01T04:16:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f08d40a2e10 msgr2=0x7f08d40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f08d40a2e10 0x7f08d40a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006010 tx=0x7f08e0009820 comp rx=0 tx=0).stop 2024-08-01T04:16:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 msgr2=0x7f08ec1a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec1a3000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08e400e740 tx=0x7f08e400ec00 comp rx=0 tx=0).stop 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 shutdown_connections 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08ec10d260 0x7f08ec1a3000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f08d40a2e10 0x7f08d40a52d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08ec108b30 0x7f08ec19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 --2- 172.21.15.3:0/3889853410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08ec108180 0x7f08ec19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 >> 172.21.15.3:0/3889853410 conn(0x7f08ec0778d0 msgr2=0x7f08ec078ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 shutdown_connections 2024-08-01T04:16:38.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:38.493+0000 7f08f4600640 1 -- 172.21.15.3:0/3889853410 wait complete. 2024-08-01T04:16:39.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:38 smithi163 ceph-mon[37383]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-08-01T04:16:39.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:38 smithi163 ceph-mon[37383]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:39.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:38 smithi003 ceph-mon[32262]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-08-01T04:16:39.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:38 smithi003 ceph-mon[32262]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:38 smithi160 ceph-mon[37224]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-08-01T04:16:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:38 smithi160 ceph-mon[37224]: from='client.25280 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-01T04:16:39.445 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-08-01T04:16:37.870912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389845,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1507,"num_read_kb":1185,"num_write":631,"num_write_kb":5302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":965,"ondisk_log_size":965,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369896,"kb_used_data":12384,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571928,"statfs":{"total":1151940427776,"available":1151561654272,"internally_reserved":0,"allocated":12681216,"data_stored":10087755,"data_compressed":385011,"data_compressed_allocated":516096,"data_compressed_original":4362240,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":344,"num_read_kb":277,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551165+0000","last_change":"2024-08-01T04:13:54.524373+0000","last_active":"2024-08-01T04:16:16.551165+0000","last_peered":"2024-08-01T04:16:16.551165+0000","last_clean":"2024-08-01T04:16:16.551165+0000","last_became_active":"2024-08-01T04:13:54.524092+0000","last_became_peered":"2024-08-01T04:13:54.524092+0000","last_unstale":"2024-08-01T04:16:16.551165+0000","last_undegraded":"2024-08-01T04:16:16.551165+0000","last_fullsized":"2024-08-01T04:16:16.551165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:20:28.689947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212873+0000","last_change":"2024-08-01T04:13:54.518647+0000","last_active":"2024-08-01T04:16:16.212873+0000","last_peered":"2024-08-01T04:16:16.212873+0000","last_clean":"2024-08-01T04:16:16.212873+0000","last_became_active":"2024-08-01T04:13:54.518315+0000","last_became_peered":"2024-08-01T04:13:54.518315+0000","last_unstale":"2024-08-01T04:16:16.212873+0000","last_undegraded":"2024-08-01T04:16:16.212873+0000","last_fullsized":"2024-08-01T04:16:16.212873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:21.603990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212612+0000","last_change":"2024-08-01T04:13:54.529785+0000","last_active":"2024-08-01T04:16:16.212612+0000","last_peered":"2024-08-01T04:16:16.212612+0000","last_clean":"2024-08-01T04:16:16.212612+0000","last_became_active":"2024-08-01T04:13:54.528986+0000","last_became_peered":"2024-08-01T04:13:54.528986+0000","last_unstale":"2024-08-01T04:16:16.212612+0000","last_undegraded":"2024-08-01T04:16:16.212612+0000","last_fullsized":"2024-08-01T04:16:16.212612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:36:00.268365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840852+0000","last_change":"2024-08-01T04:14:50.045288+0000","last_active":"2024-08-01T04:16:15.840852+0000","last_peered":"2024-08-01T04:16:15.840852+0000","last_clean":"2024-08-01T04:16:15.840852+0000","last_became_active":"2024-08-01T04:14:50.044433+0000","last_became_peered":"2024-08-01T04:14:50.044433+0000","last_unstale":"2024-08-01T04:16:15.840852+0000","last_undegraded":"2024-08-01T04:16:15.840852+0000","last_fullsized":"2024-08-01T04:16:15.840852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:00:25.613420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953698+0000","last_change":"2024-08-01T04:13:54.520769+0000","last_active":"2024-08-01T04:16:15.953698+0000","last_peered":"2024-08-01T04:16:15.953698+0000","last_clean":"2024-08-01T04:16:15.953698+0000","last_became_active":"2024-08-01T04:13:54.520350+0000","last_became_peered":"2024-08-01T04:13:54.520350+0000","last_unstale":"2024-08-01T04:16:15.953698+0000","last_undegraded":"2024-08-01T04:16:15.953698+0000","last_fullsized":"2024-08-01T04:16:15.953698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:24:01.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967039+0000","last_change":"2024-08-01T04:13:54.503536+0000","last_active":"2024-08-01T04:16:15.967039+0000","last_peered":"2024-08-01T04:16:15.967039+0000","last_clean":"2024-08-01T04:16:15.967039+0000","last_became_active":"2024-08-01T04:13:54.503215+0000","last_became_peered":"2024-08-01T04:13:54.503215+0000","last_unstale":"2024-08-01T04:16:15.967039+0000","last_undegraded":"2024-08-01T04:16:15.967039+0000","last_fullsized":"2024-08-01T04:16:15.967039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:59:11.533729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210138+0000","last_change":"2024-08-01T04:13:54.511835+0000","last_active":"2024-08-01T04:16:16.210138+0000","last_peered":"2024-08-01T04:16:16.210138+0000","last_clean":"2024-08-01T04:16:16.210138+0000","last_became_active":"2024-08-01T04:13:54.511503+0000","last_became_peered":"2024-08-01T04:13:54.511503+0000","last_unstale":"2024-08-01T04:16:16.210138+0000","last_undegraded":"2024-08-01T04:16:16.210138+0000","last_fullsized":"2024-08-01T04:16:16.210138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:54:00.518643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879784+0000","last_change":"2024-08-01T04:13:54.527906+0000","last_active":"2024-08-01T04:16:15.879784+0000","last_peered":"2024-08-01T04:16:15.879784+0000","last_clean":"2024-08-01T04:16:15.879784+0000","last_became_active":"2024-08-01T04:13:54.525437+0000","last_became_peered":"2024-08-01T04:13:54.525437+0000","last_unstale":"2024-08-01T04:16:15.879784+0000","last_undegraded":"2024-08-01T04:16:15.879784+0000","last_fullsized":"2024-08-01T04:16:15.879784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:32:04.652478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006355+0000","last_change":"2024-08-01T04:13:54.504450+0000","last_active":"2024-08-01T04:16:16.006355+0000","last_peered":"2024-08-01T04:16:16.006355+0000","last_clean":"2024-08-01T04:16:16.006355+0000","last_became_active":"2024-08-01T04:13:54.504014+0000","last_became_peered":"2024-08-01T04:13:54.504014+0000","last_unstale":"2024-08-01T04:16:16.006355+0000","last_undegraded":"2024-08-01T04:16:16.006355+0000","last_fullsized":"2024-08-01T04:16:16.006355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:14:01.293986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967418+0000","last_change":"2024-08-01T04:13:54.507171+0000","last_active":"2024-08-01T04:16:15.967418+0000","last_peered":"2024-08-01T04:16:15.967418+0000","last_clean":"2024-08-01T04:16:15.967418+0000","last_became_active":"2024-08-01T04:13:54.506829+0000","last_became_peered":"2024-08-01T04:13:54.506829+0000","last_unstale":"2024-08-01T04:16:15.967418+0000","last_undegraded":"2024-08-01T04:16:15.967418+0000","last_fullsized":"2024-08-01T04:16:15.967418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:49:27.430300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210488+0000","last_change":"2024-08-01T04:13:54.507174+0000","last_active":"2024-08-01T04:16:16.210488+0000","last_peered":"2024-08-01T04:16:16.210488+0000","last_clean":"2024-08-01T04:16:16.210488+0000","last_became_active":"2024-08-01T04:13:54.506783+0000","last_became_peered":"2024-08-01T04:13:54.506783+0000","last_unstale":"2024-08-01T04:16:16.210488+0000","last_undegraded":"2024-08-01T04:16:16.210488+0000","last_fullsized":"2024-08-01T04:16:16.210488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:10:31.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841291+0000","last_change":"2024-08-01T04:13:54.503335+0000","last_active":"2024-08-01T04:16:15.841291+0000","last_peered":"2024-08-01T04:16:15.841291+0000","last_clean":"2024-08-01T04:16:15.841291+0000","last_became_active":"2024-08-01T04:13:54.502976+0000","last_became_peered":"2024-08-01T04:13:54.502976+0000","last_unstale":"2024-08-01T04:16:15.841291+0000","last_undegraded":"2024-08-01T04:16:15.841291+0000","last_fullsized":"2024-08-01T04:16:15.841291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:20:55.723789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877759+0000","last_change":"2024-08-01T04:13:54.493945+0000","last_active":"2024-08-01T04:16:15.877759+0000","last_peered":"2024-08-01T04:16:15.877759+0000","last_clean":"2024-08-01T04:16:15.877759+0000","last_became_active":"2024-08-01T04:13:54.492899+0000","last_became_peered":"2024-08-01T04:13:54.492899+0000","last_unstale":"2024-08-01T04:16:15.877759+0000","last_undegraded":"2024-08-01T04:16:15.877759+0000","last_fullsized":"2024-08-01T04:16:15.877759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:42:19.599780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211881+0000","last_change":"2024-08-01T04:13:54.523955+0000","last_active":"2024-08-01T04:16:16.211881+0000","last_peered":"2024-08-01T04:16:16.211881+0000","last_clean":"2024-08-01T04:16:16.211881+0000","last_became_active":"2024-08-01T04:13:54.522421+0000","last_became_peered":"2024-08-01T04:13:54.522421+0000","last_unstale":"2024-08-01T04:16:16.211881+0000","last_undegraded":"2024-08-01T04:16:16.211881+0000","last_fullsized":"2024-08-01T04:16:16.211881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:51:23.039669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.211595+0000","last_change":"2024-08-01T04:13:54.529277+0000","last_active":"2024-08-01T04:16:16.211595+0000","last_peered":"2024-08-01T04:16:16.211595+0000","last_clean":"2024-08-01T04:16:16.211595+0000","last_became_active":"2024-08-01T04:13:54.521914+0000","last_became_peered":"2024-08-01T04:13:54.521914+0000","last_unstale":"2024-08-01T04:16:16.211595+0000","last_undegraded":"2024-08-01T04:16:16.211595+0000","last_fullsized":"2024-08-01T04:16:16.211595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:09:05.385086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005632+0000","last_change":"2024-08-01T04:13:54.518460+0000","last_active":"2024-08-01T04:16:16.005632+0000","last_peered":"2024-08-01T04:16:16.005632+0000","last_clean":"2024-08-01T04:16:16.005632+0000","last_became_active":"2024-08-01T04:13:54.518111+0000","last_became_peered":"2024-08-01T04:13:54.518111+0000","last_unstale":"2024-08-01T04:16:16.005632+0000","last_undegraded":"2024-08-01T04:16:16.005632+0000","last_fullsized":"2024-08-01T04:16:16.005632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:35.369498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551271+0000","last_change":"2024-08-01T04:13:54.520807+0000","last_active":"2024-08-01T04:16:16.551271+0000","last_peered":"2024-08-01T04:16:16.551271+0000","last_clean":"2024-08-01T04:16:16.551271+0000","last_became_active":"2024-08-01T04:13:54.520475+0000","last_became_peered":"2024-08-01T04:13:54.520475+0000","last_unstale":"2024-08-01T04:16:16.551271+0000","last_undegraded":"2024-08-01T04:16:16.551271+0000","last_fullsized":"2024-08-01T04:16:16.551271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:07:52.608841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840840+0000","last_change":"2024-08-01T04:13:54.516195+0000","last_active":"2024-08-01T04:16:15.840840+0000","last_peered":"2024-08-01T04:16:15.840840+0000","last_clean":"2024-08-01T04:16:15.840840+0000","last_became_active":"2024-08-01T04:13:54.515695+0000","last_became_peered":"2024-08-01T04:13:54.515695+0000","last_unstale":"2024-08-01T04:16:15.840840+0000","last_undegraded":"2024-08-01T04:16:15.840840+0000","last_fullsized":"2024-08-01T04:16:15.840840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:31:02.307530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840916+0000","last_change":"2024-08-01T04:13:54.530803+0000","last_active":"2024-08-01T04:16:15.840916+0000","last_peered":"2024-08-01T04:16:15.840916+0000","last_clean":"2024-08-01T04:16:15.840916+0000","last_became_active":"2024-08-01T04:13:54.530466+0000","last_became_peered":"2024-08-01T04:13:54.530466+0000","last_unstale":"2024-08-01T04:16:15.840916+0000","last_undegraded":"2024-08-01T04:16:15.840916+0000","last_fullsized":"2024-08-01T04:16:15.840916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:37:01.901877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550965+0000","last_change":"2024-08-01T04:13:54.503547+0000","last_active":"2024-08-01T04:16:16.550965+0000","last_peered":"2024-08-01T04:16:16.550965+0000","last_clean":"2024-08-01T04:16:16.550965+0000","last_became_active":"2024-08-01T04:13:54.503236+0000","last_became_peered":"2024-08-01T04:13:54.503236+0000","last_unstale":"2024-08-01T04:16:16.550965+0000","last_undegraded":"2024-08-01T04:16:16.550965+0000","last_fullsized":"2024-08-01T04:16:16.550965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:02:09.320847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877859+0000","last_change":"2024-08-01T04:13:54.534067+0000","last_active":"2024-08-01T04:16:15.877859+0000","last_peered":"2024-08-01T04:16:15.877859+0000","last_clean":"2024-08-01T04:16:15.877859+0000","last_became_active":"2024-08-01T04:13:54.533707+0000","last_became_peered":"2024-08-01T04:13:54.533707+0000","last_unstale":"2024-08-01T04:16:15.877859+0000","last_undegraded":"2024-08-01T04:16:15.877859+0000","last_fullsized":"2024-08-01T04:16:15.877859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:36:39.112708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006020+0000","last_change":"2024-08-01T04:14:50.043134+0000","last_active":"2024-08-01T04:16:16.006020+0000","last_peered":"2024-08-01T04:16:16.006020+0000","last_clean":"2024-08-01T04:16:16.006020+0000","last_became_active":"2024-08-01T04:14:50.042803+0000","last_became_peered":"2024-08-01T04:14:50.042803+0000","last_unstale":"2024-08-01T04:16:16.006020+0000","last_undegraded":"2024-08-01T04:16:16.006020+0000","last_fullsized":"2024-08-01T04:16:16.006020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:29:17.306177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879660+0000","last_change":"2024-08-01T04:13:54.526647+0000","last_active":"2024-08-01T04:16:15.879660+0000","last_peered":"2024-08-01T04:16:15.879660+0000","last_clean":"2024-08-01T04:16:15.879660+0000","last_became_active":"2024-08-01T04:13:54.523702+0000","last_became_peered":"2024-08-01T04:13:54.523702+0000","last_unstale":"2024-08-01T04:16:15.879660+0000","last_undegraded":"2024-08-01T04:16:15.879660+0000","last_fullsized":"2024-08-01T04:16:15.879660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:58.055005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953511+0000","last_change":"2024-08-01T04:14:50.047389+0000","last_active":"2024-08-01T04:16:15.953511+0000","last_peered":"2024-08-01T04:16:15.953511+0000","last_clean":"2024-08-01T04:16:15.953511+0000","last_became_active":"2024-08-01T04:14:50.047117+0000","last_became_peered":"2024-08-01T04:14:50.047117+0000","last_unstale":"2024-08-01T04:16:15.953511+0000","last_undegraded":"2024-08-01T04:16:15.953511+0000","last_fullsized":"2024-08-01T04:16:15.953511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:01:37.926239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880212+0000","last_change":"2024-08-01T04:13:54.526558+0000","last_active":"2024-08-01T04:16:15.880212+0000","last_peered":"2024-08-01T04:16:15.880212+0000","last_clean":"2024-08-01T04:16:15.880212+0000","last_became_active":"2024-08-01T04:13:54.525043+0000","last_became_peered":"2024-08-01T04:13:54.525043+0000","last_unstale":"2024-08-01T04:16:15.880212+0000","last_undegraded":"2024-08-01T04:16:15.880212+0000","last_fullsized":"2024-08-01T04:16:15.880212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:48:01.103676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840932+0000","last_change":"2024-08-01T04:13:54.506170+0000","last_active":"2024-08-01T04:16:15.840932+0000","last_peered":"2024-08-01T04:16:15.840932+0000","last_clean":"2024-08-01T04:16:15.840932+0000","last_became_active":"2024-08-01T04:13:54.505834+0000","last_became_peered":"2024-08-01T04:13:54.505834+0000","last_unstale":"2024-08-01T04:16:15.840932+0000","last_undegraded":"2024-08-01T04:16:15.840932+0000","last_fullsized":"2024-08-01T04:16:15.840932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:33:26.474242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551039+0000","last_change":"2024-08-01T04:13:54.503910+0000","last_active":"2024-08-01T04:16:16.551039+0000","last_peered":"2024-08-01T04:16:16.551039+0000","last_clean":"2024-08-01T04:16:16.551039+0000","last_became_active":"2024-08-01T04:13:54.503586+0000","last_became_peered":"2024-08-01T04:13:54.503586+0000","last_unstale":"2024-08-01T04:16:16.551039+0000","last_undegraded":"2024-08-01T04:16:16.551039+0000","last_fullsized":"2024-08-01T04:16:16.551039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:42.486568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877756+0000","last_change":"2024-08-01T04:13:54.522794+0000","last_active":"2024-08-01T04:16:15.877756+0000","last_peered":"2024-08-01T04:16:15.877756+0000","last_clean":"2024-08-01T04:16:15.877756+0000","last_became_active":"2024-08-01T04:13:54.522425+0000","last_became_peered":"2024-08-01T04:13:54.522425+0000","last_unstale":"2024-08-01T04:16:15.877756+0000","last_undegraded":"2024-08-01T04:16:15.877756+0000","last_fullsized":"2024-08-01T04:16:15.877756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:33:50.525325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212112+0000","last_change":"2024-08-01T04:13:54.530874+0000","last_active":"2024-08-01T04:16:16.212112+0000","last_peered":"2024-08-01T04:16:16.212112+0000","last_clean":"2024-08-01T04:16:16.212112+0000","last_became_active":"2024-08-01T04:13:54.530539+0000","last_became_peered":"2024-08-01T04:13:54.530539+0000","last_unstale":"2024-08-01T04:16:16.212112+0000","last_undegraded":"2024-08-01T04:16:16.212112+0000","last_fullsized":"2024-08-01T04:16:16.212112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:35:09.379977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840499+0000","last_change":"2024-08-01T04:14:50.052215+0000","last_active":"2024-08-01T04:16:15.840499+0000","last_peered":"2024-08-01T04:16:15.840499+0000","last_clean":"2024-08-01T04:16:15.840499+0000","last_became_active":"2024-08-01T04:14:50.051735+0000","last_became_peered":"2024-08-01T04:14:50.051735+0000","last_unstale":"2024-08-01T04:16:15.840499+0000","last_undegraded":"2024-08-01T04:16:15.840499+0000","last_fullsized":"2024-08-01T04:16:15.840499+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:20:25.419348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840592+0000","last_change":"2024-08-01T04:13:54.506134+0000","last_active":"2024-08-01T04:16:15.840592+0000","last_peered":"2024-08-01T04:16:15.840592+0000","last_clean":"2024-08-01T04:16:15.840592+0000","last_became_active":"2024-08-01T04:13:54.505767+0000","last_became_peered":"2024-08-01T04:13:54.505767+0000","last_unstale":"2024-08-01T04:16:15.840592+0000","last_undegraded":"2024-08-01T04:16:15.840592+0000","last_fullsized":"2024-08-01T04:16:15.840592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:29:20.654462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550743+0000","last_change":"2024-08-01T04:13:54.507557+0000","last_active":"2024-08-01T04:16:16.550743+0000","last_peered":"2024-08-01T04:16:16.550743+0000","last_clean":"2024-08-01T04:16:16.550743+0000","last_became_active":"2024-08-01T04:13:54.506660+0000","last_became_peered":"2024-08-01T04:13:54.506660+0000","last_unstale":"2024-08-01T04:16:16.550743+0000","last_undegraded":"2024-08-01T04:16:16.550743+0000","last_fullsized":"2024-08-01T04:16:16.550743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:22:53.409803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953699+0000","last_change":"2024-08-01T04:13:54.506868+0000","last_active":"2024-08-01T04:16:15.953699+0000","last_peered":"2024-08-01T04:16:15.953699+0000","last_clean":"2024-08-01T04:16:15.953699+0000","last_became_active":"2024-08-01T04:13:54.506547+0000","last_became_peered":"2024-08-01T04:13:54.506547+0000","last_unstale":"2024-08-01T04:16:15.953699+0000","last_undegraded":"2024-08-01T04:16:15.953699+0000","last_fullsized":"2024-08-01T04:16:15.953699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:31:05.092153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":195,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:34.864727+0000","last_change":"2024-08-01T04:13:51.462744+0000","last_active":"2024-08-01T04:16:34.864727+0000","last_peered":"2024-08-01T04:16:34.864727+0000","last_clean":"2024-08-01T04:16:34.864727+0000","last_became_active":"2024-08-01T04:13:51.462366+0000","last_became_peered":"2024-08-01T04:13:51.462366+0000","last_unstale":"2024-08-01T04:16:34.864727+0000","last_undegraded":"2024-08-01T04:16:34.864727+0000","last_fullsized":"2024-08-01T04:16:34.864727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:51:32.809429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963465+0000","last_change":"2024-08-01T04:13:54.526904+0000","last_active":"2024-08-01T04:16:15.963465+0000","last_peered":"2024-08-01T04:16:15.963465+0000","last_clean":"2024-08-01T04:16:15.963465+0000","last_became_active":"2024-08-01T04:13:54.526520+0000","last_became_peered":"2024-08-01T04:13:54.526520+0000","last_unstale":"2024-08-01T04:16:15.963465+0000","last_undegraded":"2024-08-01T04:16:15.963465+0000","last_fullsized":"2024-08-01T04:16:15.963465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:44:11.668476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967109+0000","last_change":"2024-08-01T04:14:50.048326+0000","last_active":"2024-08-01T04:16:15.967109+0000","last_peered":"2024-08-01T04:16:15.967109+0000","last_clean":"2024-08-01T04:16:15.967109+0000","last_became_active":"2024-08-01T04:14:50.040820+0000","last_became_peered":"2024-08-01T04:14:50.040820+0000","last_unstale":"2024-08-01T04:16:15.967109+0000","last_undegraded":"2024-08-01T04:16:15.967109+0000","last_fullsized":"2024-08-01T04:16:15.967109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T16:10:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877210+0000","last_change":"2024-08-01T04:13:51.465994+0000","last_active":"2024-08-01T04:16:15.877210+0000","last_peered":"2024-08-01T04:16:15.877210+0000","last_clean":"2024-08-01T04:16:15.877210+0000","last_became_active":"2024-08-01T04:13:51.465542+0000","last_became_peered":"2024-08-01T04:13:51.465542+0000","last_unstale":"2024-08-01T04:16:15.877210+0000","last_undegraded":"2024-08-01T04:16:15.877210+0000","last_fullsized":"2024-08-01T04:16:15.877210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:42.663466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879869+0000","last_change":"2024-08-01T04:13:54.520427+0000","last_active":"2024-08-01T04:16:15.879869+0000","last_peered":"2024-08-01T04:16:15.879869+0000","last_clean":"2024-08-01T04:16:15.879869+0000","last_became_active":"2024-08-01T04:13:54.520033+0000","last_became_peered":"2024-08-01T04:13:54.520033+0000","last_unstale":"2024-08-01T04:16:15.879869+0000","last_undegraded":"2024-08-01T04:16:15.879869+0000","last_fullsized":"2024-08-01T04:16:15.879869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:48:33.702234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963453+0000","last_change":"2024-08-01T04:13:51.464010+0000","last_active":"2024-08-01T04:16:15.963453+0000","last_peered":"2024-08-01T04:16:15.963453+0000","last_clean":"2024-08-01T04:16:15.963453+0000","last_became_active":"2024-08-01T04:13:51.463641+0000","last_became_peered":"2024-08-01T04:13:51.463641+0000","last_unstale":"2024-08-01T04:16:15.963453+0000","last_undegraded":"2024-08-01T04:16:15.963453+0000","last_fullsized":"2024-08-01T04:16:15.963453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:57:34.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840696+0000","last_change":"2024-08-01T04:13:54.526317+0000","last_active":"2024-08-01T04:16:15.840696+0000","last_peered":"2024-08-01T04:16:15.840696+0000","last_clean":"2024-08-01T04:16:15.840696+0000","last_became_active":"2024-08-01T04:13:54.526003+0000","last_became_peered":"2024-08-01T04:13:54.526003+0000","last_unstale":"2024-08-01T04:16:15.840696+0000","last_undegraded":"2024-08-01T04:16:15.840696+0000","last_fullsized":"2024-08-01T04:16:15.840696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:11:21.042600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":185,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:35.865914+0000","last_change":"2024-08-01T04:13:51.468359+0000","last_active":"2024-08-01T04:16:35.865914+0000","last_peered":"2024-08-01T04:16:35.865914+0000","last_clean":"2024-08-01T04:16:35.865914+0000","last_became_active":"2024-08-01T04:13:51.468041+0000","last_became_peered":"2024-08-01T04:13:51.468041+0000","last_unstale":"2024-08-01T04:16:35.865914+0000","last_undegraded":"2024-08-01T04:16:35.865914+0000","last_fullsized":"2024-08-01T04:16:35.865914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:54.443954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006173+0000","last_change":"2024-08-01T04:13:54.514871+0000","last_active":"2024-08-01T04:16:16.006173+0000","last_peered":"2024-08-01T04:16:16.006173+0000","last_clean":"2024-08-01T04:16:16.006173+0000","last_became_active":"2024-08-01T04:13:54.514507+0000","last_became_peered":"2024-08-01T04:13:54.514507+0000","last_unstale":"2024-08-01T04:16:16.006173+0000","last_undegraded":"2024-08-01T04:16:16.006173+0000","last_fullsized":"2024-08-01T04:16:16.006173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:05:35.086163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210517+0000","last_change":"2024-08-01T04:13:51.458735+0000","last_active":"2024-08-01T04:16:16.210517+0000","last_peered":"2024-08-01T04:16:16.210517+0000","last_clean":"2024-08-01T04:16:16.210517+0000","last_became_active":"2024-08-01T04:13:51.458233+0000","last_became_peered":"2024-08-01T04:13:51.458233+0000","last_unstale":"2024-08-01T04:16:16.210517+0000","last_undegraded":"2024-08-01T04:16:16.210517+0000","last_fullsized":"2024-08-01T04:16:16.210517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:09:26.554152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213392+0000","last_change":"2024-08-01T04:13:54.528961+0000","last_active":"2024-08-01T04:16:16.213392+0000","last_peered":"2024-08-01T04:16:16.213392+0000","last_clean":"2024-08-01T04:16:16.213392+0000","last_became_active":"2024-08-01T04:13:54.522775+0000","last_became_peered":"2024-08-01T04:13:54.522775+0000","last_unstale":"2024-08-01T04:16:16.213392+0000","last_undegraded":"2024-08-01T04:16:16.213392+0000","last_fullsized":"2024-08-01T04:16:16.213392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:28:32.085654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963763+0000","last_change":"2024-08-01T04:13:51.578797+0000","last_active":"2024-08-01T04:16:15.963763+0000","last_peered":"2024-08-01T04:16:15.963763+0000","last_clean":"2024-08-01T04:16:15.963763+0000","last_became_active":"2024-08-01T04:13:51.578453+0000","last_became_peered":"2024-08-01T04:13:51.578453+0000","last_unstale":"2024-08-01T04:16:15.963763+0000","last_undegraded":"2024-08-01T04:16:15.963763+0000","last_fullsized":"2024-08-01T04:16:15.963763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:11:30.198919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.006056+0000","last_change":"2024-08-01T04:14:50.044245+0000","last_active":"2024-08-01T04:16:16.006056+0000","last_peered":"2024-08-01T04:16:16.006056+0000","last_clean":"2024-08-01T04:16:16.006056+0000","last_became_active":"2024-08-01T04:14:50.043759+0000","last_became_peered":"2024-08-01T04:14:50.043759+0000","last_unstale":"2024-08-01T04:16:16.006056+0000","last_undegraded":"2024-08-01T04:16:16.006056+0000","last_fullsized":"2024-08-01T04:16:16.006056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:21:03.068138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":378,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:36.867401+0000","last_change":"2024-08-01T04:13:51.458233+0000","last_active":"2024-08-01T04:16:36.867401+0000","last_peered":"2024-08-01T04:16:36.867401+0000","last_clean":"2024-08-01T04:16:36.867401+0000","last_became_active":"2024-08-01T04:13:51.457660+0000","last_became_peered":"2024-08-01T04:13:51.457660+0000","last_unstale":"2024-08-01T04:16:36.867401+0000","last_undegraded":"2024-08-01T04:16:36.867401+0000","last_fullsized":"2024-08-01T04:16:36.867401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:34:21.778221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71563,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":309,"num_read_kb":224,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954043+0000","last_change":"2024-08-01T04:13:54.508324+0000","last_active":"2024-08-01T04:16:15.954043+0000","last_peered":"2024-08-01T04:16:15.954043+0000","last_clean":"2024-08-01T04:16:15.954043+0000","last_became_active":"2024-08-01T04:13:54.507964+0000","last_became_peered":"2024-08-01T04:13:54.507964+0000","last_unstale":"2024-08-01T04:16:15.954043+0000","last_undegraded":"2024-08-01T04:16:15.954043+0000","last_fullsized":"2024-08-01T04:16:15.954043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:52:09.975881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.840839+0000","last_change":"2024-08-01T04:13:51.465213+0000","last_active":"2024-08-01T04:16:15.840839+0000","last_peered":"2024-08-01T04:16:15.840839+0000","last_clean":"2024-08-01T04:16:15.840839+0000","last_became_active":"2024-08-01T04:13:51.464859+0000","last_became_peered":"2024-08-01T04:13:51.464859+0000","last_unstale":"2024-08-01T04:16:15.840839+0000","last_undegraded":"2024-08-01T04:16:15.840839+0000","last_fullsized":"2024-08-01T04:16:15.840839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T13:37:22.501064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877540+0000","last_change":"2024-08-01T04:13:54.528975+0000","last_active":"2024-08-01T04:16:15.877540+0000","last_peered":"2024-08-01T04:16:15.877540+0000","last_clean":"2024-08-01T04:16:15.877540+0000","last_became_active":"2024-08-01T04:13:54.528602+0000","last_became_peered":"2024-08-01T04:13:54.528602+0000","last_unstale":"2024-08-01T04:16:15.877540+0000","last_undegraded":"2024-08-01T04:16:15.877540+0000","last_fullsized":"2024-08-01T04:16:15.877540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:26:50.905985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963535+0000","last_change":"2024-08-01T04:13:51.464976+0000","last_active":"2024-08-01T04:16:15.963535+0000","last_peered":"2024-08-01T04:16:15.963535+0000","last_clean":"2024-08-01T04:16:15.963535+0000","last_became_active":"2024-08-01T04:13:51.464570+0000","last_became_peered":"2024-08-01T04:13:51.464570+0000","last_unstale":"2024-08-01T04:16:15.963535+0000","last_undegraded":"2024-08-01T04:16:15.963535+0000","last_fullsized":"2024-08-01T04:16:15.963535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:02:32.285376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210232+0000","last_change":"2024-08-01T04:13:54.512007+0000","last_active":"2024-08-01T04:16:16.210232+0000","last_peered":"2024-08-01T04:16:16.210232+0000","last_clean":"2024-08-01T04:16:16.210232+0000","last_became_active":"2024-08-01T04:13:54.511629+0000","last_became_peered":"2024-08-01T04:13:54.511629+0000","last_unstale":"2024-08-01T04:16:16.210232+0000","last_undegraded":"2024-08-01T04:16:16.210232+0000","last_fullsized":"2024-08-01T04:16:16.210232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:34:16.697925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":192,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:34.920464+0000","last_change":"2024-08-01T04:13:51.461406+0000","last_active":"2024-08-01T04:16:34.920464+0000","last_peered":"2024-08-01T04:16:34.920464+0000","last_clean":"2024-08-01T04:16:34.920464+0000","last_became_active":"2024-08-01T04:13:51.460759+0000","last_became_peered":"2024-08-01T04:13:51.460759+0000","last_unstale":"2024-08-01T04:16:34.920464+0000","last_undegraded":"2024-08-01T04:16:34.920464+0000","last_fullsized":"2024-08-01T04:16:34.920464+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:45:38.257285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210072+0000","last_change":"2024-08-01T04:14:50.044294+0000","last_active":"2024-08-01T04:16:16.210072+0000","last_peered":"2024-08-01T04:16:16.210072+0000","last_clean":"2024-08-01T04:16:16.210072+0000","last_became_active":"2024-08-01T04:14:50.043911+0000","last_became_peered":"2024-08-01T04:14:50.043911+0000","last_unstale":"2024-08-01T04:16:16.210072+0000","last_undegraded":"2024-08-01T04:16:16.210072+0000","last_fullsized":"2024-08-01T04:16:16.210072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:32:27.860163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953840+0000","last_change":"2024-08-01T04:13:51.578895+0000","last_active":"2024-08-01T04:16:15.953840+0000","last_peered":"2024-08-01T04:16:15.953840+0000","last_clean":"2024-08-01T04:16:15.953840+0000","last_became_active":"2024-08-01T04:13:51.578533+0000","last_became_peered":"2024-08-01T04:13:51.578533+0000","last_unstale":"2024-08-01T04:16:15.953840+0000","last_undegraded":"2024-08-01T04:16:15.953840+0000","last_fullsized":"2024-08-01T04:16:15.953840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T10:00:52.664173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963507+0000","last_change":"2024-08-01T04:13:54.504959+0000","last_active":"2024-08-01T04:16:15.963507+0000","last_peered":"2024-08-01T04:16:15.963507+0000","last_clean":"2024-08-01T04:16:15.963507+0000","last_became_active":"2024-08-01T04:13:54.504524+0000","last_became_peered":"2024-08-01T04:13:54.504524+0000","last_unstale":"2024-08-01T04:16:15.963507+0000","last_undegraded":"2024-08-01T04:16:15.963507+0000","last_fullsized":"2024-08-01T04:16:15.963507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:56.252067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954076+0000","last_change":"2024-08-01T04:13:51.459499+0000","last_active":"2024-08-01T04:16:15.954076+0000","last_peered":"2024-08-01T04:16:15.954076+0000","last_clean":"2024-08-01T04:16:15.954076+0000","last_became_active":"2024-08-01T04:13:51.459050+0000","last_became_peered":"2024-08-01T04:13:51.459050+0000","last_unstale":"2024-08-01T04:16:15.954076+0000","last_undegraded":"2024-08-01T04:16:15.954076+0000","last_fullsized":"2024-08-01T04:16:15.954076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:43:15.959683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.551119+0000","last_change":"2024-08-01T04:13:54.502098+0000","last_active":"2024-08-01T04:16:16.551119+0000","last_peered":"2024-08-01T04:16:16.551119+0000","last_clean":"2024-08-01T04:16:16.551119+0000","last_became_active":"2024-08-01T04:13:54.501797+0000","last_became_peered":"2024-08-01T04:13:54.501797+0000","last_unstale":"2024-08-01T04:16:16.551119+0000","last_undegraded":"2024-08-01T04:16:16.551119+0000","last_fullsized":"2024-08-01T04:16:16.551119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:00:20.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877775+0000","last_change":"2024-08-01T04:15:50.404577+0000","last_active":"2024-08-01T04:16:15.877775+0000","last_peered":"2024-08-01T04:16:15.877775+0000","last_clean":"2024-08-01T04:16:15.877775+0000","last_became_active":"2024-08-01T04:15:50.400018+0000","last_became_peered":"2024-08-01T04:15:50.400018+0000","last_unstale":"2024-08-01T04:16:15.877775+0000","last_undegraded":"2024-08-01T04:16:15.877775+0000","last_fullsized":"2024-08-01T04:16:15.877775+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:22:39.045462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210494+0000","last_change":"2024-08-01T04:13:54.509439+0000","last_active":"2024-08-01T04:16:16.210494+0000","last_peered":"2024-08-01T04:16:16.210494+0000","last_clean":"2024-08-01T04:16:16.210494+0000","last_became_active":"2024-08-01T04:13:54.509105+0000","last_became_peered":"2024-08-01T04:13:54.509105+0000","last_unstale":"2024-08-01T04:16:16.210494+0000","last_undegraded":"2024-08-01T04:16:16.210494+0000","last_fullsized":"2024-08-01T04:16:16.210494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:59:47.364236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'65","reported_seq":129,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:31.460894+0000","last_change":"2024-08-01T04:13:51.469252+0000","last_active":"2024-08-01T04:16:31.460894+0000","last_peered":"2024-08-01T04:16:31.460894+0000","last_clean":"2024-08-01T04:16:31.460894+0000","last_became_active":"2024-08-01T04:13:51.468861+0000","last_became_peered":"2024-08-01T04:13:51.468861+0000","last_unstale":"2024-08-01T04:16:31.460894+0000","last_undegraded":"2024-08-01T04:16:31.460894+0000","last_fullsized":"2024-08-01T04:16:31.460894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:45:16.482202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210202+0000","last_change":"2024-08-01T04:13:54.520812+0000","last_active":"2024-08-01T04:16:16.210202+0000","last_peered":"2024-08-01T04:16:16.210202+0000","last_clean":"2024-08-01T04:16:16.210202+0000","last_became_active":"2024-08-01T04:13:54.520525+0000","last_became_peered":"2024-08-01T04:13:54.520525+0000","last_unstale":"2024-08-01T04:16:16.210202+0000","last_undegraded":"2024-08-01T04:16:16.210202+0000","last_fullsized":"2024-08-01T04:16:16.210202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:09:06.932000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841059+0000","last_change":"2024-08-01T04:13:51.579171+0000","last_active":"2024-08-01T04:16:15.841059+0000","last_peered":"2024-08-01T04:16:15.841059+0000","last_clean":"2024-08-01T04:16:15.841059+0000","last_became_active":"2024-08-01T04:13:51.578278+0000","last_became_peered":"2024-08-01T04:13:51.578278+0000","last_unstale":"2024-08-01T04:16:15.841059+0000","last_undegraded":"2024-08-01T04:16:15.841059+0000","last_fullsized":"2024-08-01T04:16:15.841059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:51:17.789607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210445+0000","last_change":"2024-08-01T04:13:54.505483+0000","last_active":"2024-08-01T04:16:16.210445+0000","last_peered":"2024-08-01T04:16:16.210445+0000","last_clean":"2024-08-01T04:16:16.210445+0000","last_became_active":"2024-08-01T04:13:54.505091+0000","last_became_peered":"2024-08-01T04:13:54.505091+0000","last_unstale":"2024-08-01T04:16:16.210445+0000","last_undegraded":"2024-08-01T04:16:16.210445+0000","last_fullsized":"2024-08-01T04:16:16.210445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:33:30.441888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":404,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:17.968134+0000","last_change":"2024-08-01T04:12:34.784580+0000","last_active":"2024-08-01T04:16:17.968134+0000","last_peered":"2024-08-01T04:16:17.968134+0000","last_clean":"2024-08-01T04:16:17.968134+0000","last_became_active":"2024-08-01T04:12:34.784095+0000","last_became_peered":"2024-08-01T04:12:34.784095+0000","last_unstale":"2024-08-01T04:16:17.968134+0000","last_undegraded":"2024-08-01T04:16:17.968134+0000","last_fullsized":"2024-08-01T04:16:17.968134+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:09:04.723926+0000","last_clean_scrub_stamp":"2024-08-01T04:09:04.723926+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:17:01.886101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841282+0000","last_change":"2024-08-01T04:13:51.467259+0000","last_active":"2024-08-01T04:16:15.841282+0000","last_peered":"2024-08-01T04:16:15.841282+0000","last_clean":"2024-08-01T04:16:15.841282+0000","last_became_active":"2024-08-01T04:13:51.466941+0000","last_became_peered":"2024-08-01T04:13:51.466941+0000","last_unstale":"2024-08-01T04:16:15.841282+0000","last_undegraded":"2024-08-01T04:16:15.841282+0000","last_fullsized":"2024-08-01T04:16:15.841282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:52:06.994493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880005+0000","last_change":"2024-08-01T04:13:54.521996+0000","last_active":"2024-08-01T04:16:15.880005+0000","last_peered":"2024-08-01T04:16:15.880005+0000","last_clean":"2024-08-01T04:16:15.880005+0000","last_became_active":"2024-08-01T04:13:54.521551+0000","last_became_peered":"2024-08-01T04:13:54.521551+0000","last_unstale":"2024-08-01T04:16:15.880005+0000","last_undegraded":"2024-08-01T04:16:15.880005+0000","last_fullsized":"2024-08-01T04:16:15.880005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:52:51.096798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210288+0000","last_change":"2024-08-01T04:13:51.461081+0000","last_active":"2024-08-01T04:16:16.210288+0000","last_peered":"2024-08-01T04:16:16.210288+0000","last_clean":"2024-08-01T04:16:16.210288+0000","last_became_active":"2024-08-01T04:13:51.460734+0000","last_became_peered":"2024-08-01T04:13:51.460734+0000","last_unstale":"2024-08-01T04:16:16.210288+0000","last_undegraded":"2024-08-01T04:16:16.210288+0000","last_fullsized":"2024-08-01T04:16:16.210288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:11:02.009646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877149+0000","last_change":"2024-08-01T04:13:54.509890+0000","last_active":"2024-08-01T04:16:15.877149+0000","last_peered":"2024-08-01T04:16:15.877149+0000","last_clean":"2024-08-01T04:16:15.877149+0000","last_became_active":"2024-08-01T04:13:54.508834+0000","last_became_peered":"2024-08-01T04:13:54.508834+0000","last_unstale":"2024-08-01T04:16:15.877149+0000","last_undegraded":"2024-08-01T04:16:15.877149+0000","last_fullsized":"2024-08-01T04:16:15.877149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:34:23.699427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":186,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:34.919759+0000","last_change":"2024-08-01T04:13:51.580080+0000","last_active":"2024-08-01T04:16:34.919759+0000","last_peered":"2024-08-01T04:16:34.919759+0000","last_clean":"2024-08-01T04:16:34.919759+0000","last_became_active":"2024-08-01T04:13:51.579628+0000","last_became_peered":"2024-08-01T04:13:51.579628+0000","last_unstale":"2024-08-01T04:16:34.919759+0000","last_undegraded":"2024-08-01T04:16:34.919759+0000","last_fullsized":"2024-08-01T04:16:34.919759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:01:18.468447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963269+0000","last_change":"2024-08-01T04:13:54.507639+0000","last_active":"2024-08-01T04:16:15.963269+0000","last_peered":"2024-08-01T04:16:15.963269+0000","last_clean":"2024-08-01T04:16:15.963269+0000","last_became_active":"2024-08-01T04:13:54.507333+0000","last_became_peered":"2024-08-01T04:13:54.507333+0000","last_unstale":"2024-08-01T04:16:15.963269+0000","last_undegraded":"2024-08-01T04:16:15.963269+0000","last_fullsized":"2024-08-01T04:16:15.963269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:21:36.219544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210461+0000","last_change":"2024-08-01T04:13:51.459605+0000","last_active":"2024-08-01T04:16:16.210461+0000","last_peered":"2024-08-01T04:16:16.210461+0000","last_clean":"2024-08-01T04:16:16.210461+0000","last_became_active":"2024-08-01T04:13:51.459275+0000","last_became_peered":"2024-08-01T04:13:51.459275+0000","last_unstale":"2024-08-01T04:16:16.210461+0000","last_undegraded":"2024-08-01T04:16:16.210461+0000","last_fullsized":"2024-08-01T04:16:16.210461+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:26:51.949371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967176+0000","last_change":"2024-08-01T04:13:54.532966+0000","last_active":"2024-08-01T04:16:15.967176+0000","last_peered":"2024-08-01T04:16:15.967176+0000","last_clean":"2024-08-01T04:16:15.967176+0000","last_became_active":"2024-08-01T04:13:54.532623+0000","last_became_peered":"2024-08-01T04:13:54.532623+0000","last_unstale":"2024-08-01T04:16:15.967176+0000","last_undegraded":"2024-08-01T04:16:15.967176+0000","last_fullsized":"2024-08-01T04:16:15.967176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:43:50.720730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841074+0000","last_change":"2024-08-01T04:13:51.583642+0000","last_active":"2024-08-01T04:16:15.841074+0000","last_peered":"2024-08-01T04:16:15.841074+0000","last_clean":"2024-08-01T04:16:15.841074+0000","last_became_active":"2024-08-01T04:13:51.580510+0000","last_became_peered":"2024-08-01T04:13:51.580510+0000","last_unstale":"2024-08-01T04:16:15.841074+0000","last_undegraded":"2024-08-01T04:16:15.841074+0000","last_fullsized":"2024-08-01T04:16:15.841074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:20:25.006998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.967171+0000","last_change":"2024-08-01T04:13:54.521595+0000","last_active":"2024-08-01T04:16:15.967171+0000","last_peered":"2024-08-01T04:16:15.967171+0000","last_clean":"2024-08-01T04:16:15.967171+0000","last_became_active":"2024-08-01T04:13:54.521305+0000","last_became_peered":"2024-08-01T04:13:54.521305+0000","last_unstale":"2024-08-01T04:16:15.967171+0000","last_undegraded":"2024-08-01T04:16:15.967171+0000","last_fullsized":"2024-08-01T04:16:15.967171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:09:39.112857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.954025+0000","last_change":"2024-08-01T04:13:51.578750+0000","last_active":"2024-08-01T04:16:15.954025+0000","last_peered":"2024-08-01T04:16:15.954025+0000","last_clean":"2024-08-01T04:16:15.954025+0000","last_became_active":"2024-08-01T04:13:51.578392+0000","last_became_peered":"2024-08-01T04:13:51.578392+0000","last_unstale":"2024-08-01T04:16:15.954025+0000","last_undegraded":"2024-08-01T04:16:15.954025+0000","last_fullsized":"2024-08-01T04:16:15.954025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T04:14:15.851438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.877800+0000","last_change":"2024-08-01T04:13:54.518110+0000","last_active":"2024-08-01T04:16:15.877800+0000","last_peered":"2024-08-01T04:16:15.877800+0000","last_clean":"2024-08-01T04:16:15.877800+0000","last_became_active":"2024-08-01T04:13:54.516631+0000","last_became_peered":"2024-08-01T04:13:54.516631+0000","last_unstale":"2024-08-01T04:16:15.877800+0000","last_undegraded":"2024-08-01T04:16:15.877800+0000","last_fullsized":"2024-08-01T04:16:15.877800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:28:16.716277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'102","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:31.936731+0000","last_change":"2024-08-01T04:13:51.470998+0000","last_active":"2024-08-01T04:16:31.936731+0000","last_peered":"2024-08-01T04:16:31.936731+0000","last_clean":"2024-08-01T04:16:31.936731+0000","last_became_active":"2024-08-01T04:13:51.470569+0000","last_became_peered":"2024-08-01T04:13:51.470569+0000","last_unstale":"2024-08-01T04:16:31.936731+0000","last_undegraded":"2024-08-01T04:16:31.936731+0000","last_fullsized":"2024-08-01T04:16:31.936731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:36:25.213653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880558+0000","last_change":"2024-08-01T04:13:54.523987+0000","last_active":"2024-08-01T04:16:15.880558+0000","last_peered":"2024-08-01T04:16:15.880558+0000","last_clean":"2024-08-01T04:16:15.880558+0000","last_became_active":"2024-08-01T04:13:54.523677+0000","last_became_peered":"2024-08-01T04:13:54.523677+0000","last_unstale":"2024-08-01T04:16:15.880558+0000","last_undegraded":"2024-08-01T04:16:15.880558+0000","last_fullsized":"2024-08-01T04:16:15.880558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:18:48.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841491+0000","last_change":"2024-08-01T04:13:51.460780+0000","last_active":"2024-08-01T04:16:15.841491+0000","last_peered":"2024-08-01T04:16:15.841491+0000","last_clean":"2024-08-01T04:16:15.841491+0000","last_became_active":"2024-08-01T04:13:51.460308+0000","last_became_peered":"2024-08-01T04:13:51.460308+0000","last_unstale":"2024-08-01T04:16:15.841491+0000","last_undegraded":"2024-08-01T04:16:15.841491+0000","last_fullsized":"2024-08-01T04:16:15.841491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:21:06.970323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.212996+0000","last_change":"2024-08-01T04:14:50.051995+0000","last_active":"2024-08-01T04:16:16.212996+0000","last_peered":"2024-08-01T04:16:16.212996+0000","last_clean":"2024-08-01T04:16:16.212996+0000","last_became_active":"2024-08-01T04:14:50.051478+0000","last_became_peered":"2024-08-01T04:14:50.051478+0000","last_unstale":"2024-08-01T04:16:16.212996+0000","last_undegraded":"2024-08-01T04:16:16.212996+0000","last_fullsized":"2024-08-01T04:16:16.212996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:15.749654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005685+0000","last_change":"2024-08-01T04:13:51.575066+0000","last_active":"2024-08-01T04:16:16.005685+0000","last_peered":"2024-08-01T04:16:16.005685+0000","last_clean":"2024-08-01T04:16:16.005685+0000","last_became_active":"2024-08-01T04:13:51.574649+0000","last_became_peered":"2024-08-01T04:13:51.574649+0000","last_unstale":"2024-08-01T04:16:16.005685+0000","last_undegraded":"2024-08-01T04:16:16.005685+0000","last_fullsized":"2024-08-01T04:16:16.005685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:19:25.517760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953515+0000","last_change":"2024-08-01T04:13:54.502286+0000","last_active":"2024-08-01T04:16:15.953515+0000","last_peered":"2024-08-01T04:16:15.953515+0000","last_clean":"2024-08-01T04:16:15.953515+0000","last_became_active":"2024-08-01T04:13:54.501944+0000","last_became_peered":"2024-08-01T04:13:54.501944+0000","last_unstale":"2024-08-01T04:16:15.953515+0000","last_undegraded":"2024-08-01T04:16:15.953515+0000","last_fullsized":"2024-08-01T04:16:15.953515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:52:43.545209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.879984+0000","last_change":"2024-08-01T04:13:51.467483+0000","last_active":"2024-08-01T04:16:15.879984+0000","last_peered":"2024-08-01T04:16:15.879984+0000","last_clean":"2024-08-01T04:16:15.879984+0000","last_became_active":"2024-08-01T04:13:51.466947+0000","last_became_peered":"2024-08-01T04:13:51.466947+0000","last_unstale":"2024-08-01T04:16:15.879984+0000","last_undegraded":"2024-08-01T04:16:15.879984+0000","last_fullsized":"2024-08-01T04:16:15.879984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:14:36.151526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550822+0000","last_change":"2024-08-01T04:13:54.520843+0000","last_active":"2024-08-01T04:16:16.550822+0000","last_peered":"2024-08-01T04:16:16.550822+0000","last_clean":"2024-08-01T04:16:16.550822+0000","last_became_active":"2024-08-01T04:13:54.520497+0000","last_became_peered":"2024-08-01T04:13:54.520497+0000","last_unstale":"2024-08-01T04:16:16.550822+0000","last_undegraded":"2024-08-01T04:16:16.550822+0000","last_fullsized":"2024-08-01T04:16:16.550822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T14:37:33.497275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550847+0000","last_change":"2024-08-01T04:14:50.051986+0000","last_active":"2024-08-01T04:16:16.550847+0000","last_peered":"2024-08-01T04:16:16.550847+0000","last_clean":"2024-08-01T04:16:16.550847+0000","last_became_active":"2024-08-01T04:14:50.048843+0000","last_became_peered":"2024-08-01T04:14:50.048843+0000","last_unstale":"2024-08-01T04:16:16.550847+0000","last_undegraded":"2024-08-01T04:16:16.550847+0000","last_fullsized":"2024-08-01T04:16:16.550847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:23:55.231083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880233+0000","last_change":"2024-08-01T04:13:54.534795+0000","last_active":"2024-08-01T04:16:15.880233+0000","last_peered":"2024-08-01T04:16:15.880233+0000","last_clean":"2024-08-01T04:16:15.880233+0000","last_became_active":"2024-08-01T04:13:54.534427+0000","last_became_peered":"2024-08-01T04:13:54.534427+0000","last_unstale":"2024-08-01T04:16:15.880233+0000","last_undegraded":"2024-08-01T04:16:15.880233+0000","last_fullsized":"2024-08-01T04:16:15.880233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T05:40:41.366669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953606+0000","last_change":"2024-08-01T04:13:51.461345+0000","last_active":"2024-08-01T04:16:15.953606+0000","last_peered":"2024-08-01T04:16:15.953606+0000","last_clean":"2024-08-01T04:16:15.953606+0000","last_became_active":"2024-08-01T04:13:51.460941+0000","last_became_peered":"2024-08-01T04:13:51.460941+0000","last_unstale":"2024-08-01T04:16:15.953606+0000","last_undegraded":"2024-08-01T04:16:15.953606+0000","last_fullsized":"2024-08-01T04:16:15.953606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T12:54:08.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.210241+0000","last_change":"2024-08-01T04:13:54.511539+0000","last_active":"2024-08-01T04:16:16.210241+0000","last_peered":"2024-08-01T04:16:16.210241+0000","last_clean":"2024-08-01T04:16:16.210241+0000","last_became_active":"2024-08-01T04:13:54.511193+0000","last_became_peered":"2024-08-01T04:13:54.511193+0000","last_unstale":"2024-08-01T04:16:16.210241+0000","last_undegraded":"2024-08-01T04:16:16.210241+0000","last_fullsized":"2024-08-01T04:16:16.210241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T09:30:14.808604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.963695+0000","last_change":"2024-08-01T04:14:50.037426+0000","last_active":"2024-08-01T04:16:15.963695+0000","last_peered":"2024-08-01T04:16:15.963695+0000","last_clean":"2024-08-01T04:16:15.963695+0000","last_became_active":"2024-08-01T04:14:50.037040+0000","last_became_peered":"2024-08-01T04:14:50.037040+0000","last_unstale":"2024-08-01T04:16:15.963695+0000","last_undegraded":"2024-08-01T04:16:15.963695+0000","last_fullsized":"2024-08-01T04:16:15.963695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:15:18.590636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.953907+0000","last_change":"2024-08-01T04:13:51.459951+0000","last_active":"2024-08-01T04:16:15.953907+0000","last_peered":"2024-08-01T04:16:15.953907+0000","last_clean":"2024-08-01T04:16:15.953907+0000","last_became_active":"2024-08-01T04:13:51.459602+0000","last_became_peered":"2024-08-01T04:13:51.459602+0000","last_unstale":"2024-08-01T04:16:15.953907+0000","last_undegraded":"2024-08-01T04:16:15.953907+0000","last_fullsized":"2024-08-01T04:16:15.953907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:21:35.294472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.005728+0000","last_change":"2024-08-01T04:13:54.502472+0000","last_active":"2024-08-01T04:16:16.005728+0000","last_peered":"2024-08-01T04:16:16.005728+0000","last_clean":"2024-08-01T04:16:16.005728+0000","last_became_active":"2024-08-01T04:13:54.502119+0000","last_became_peered":"2024-08-01T04:13:54.502119+0000","last_unstale":"2024-08-01T04:16:16.005728+0000","last_undegraded":"2024-08-01T04:16:16.005728+0000","last_fullsized":"2024-08-01T04:16:16.005728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T08:16:18.227396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.550806+0000","last_change":"2024-08-01T04:15:50.403853+0000","last_active":"2024-08-01T04:16:16.550806+0000","last_peered":"2024-08-01T04:16:16.550806+0000","last_clean":"2024-08-01T04:16:16.550806+0000","last_became_active":"2024-08-01T04:15:50.400282+0000","last_became_peered":"2024-08-01T04:15:50.400282+0000","last_unstale":"2024-08-01T04:16:16.550806+0000","last_undegraded":"2024-08-01T04:16:16.550806+0000","last_fullsized":"2024-08-01T04:16:16.550806+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T11:11:52.649176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:16.213209+0000","last_change":"2024-08-01T04:13:54.526065+0000","last_active":"2024-08-01T04:16:16.213209+0000","last_peered":"2024-08-01T04:16:16.213209+0000","last_clean":"2024-08-01T04:16:16.213209+0000","last_became_active":"2024-08-01T04:13:54.522944+0000","last_became_peered":"2024-08-01T04:13:54.522944+0000","last_unstale":"2024-08-01T04:16:16.213209+0000","last_undegraded":"2024-08-01T04:16:16.213209+0000","last_fullsized":"2024-08-01T04:16:16.213209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T15:10:15.734732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880724+0000","last_change":"2024-08-01T04:13:51.580077+0000","last_active":"2024-08-01T04:16:15.880724+0000","last_peered":"2024-08-01T04:16:15.880724+0000","last_clean":"2024-08-01T04:16:15.880724+0000","last_became_active":"2024-08-01T04:13:51.579100+0000","last_became_peered":"2024-08-01T04:13:51.579100+0000","last_unstale":"2024-08-01T04:16:15.880724+0000","last_undegraded":"2024-08-01T04:16:15.880724+0000","last_fullsized":"2024-08-01T04:16:15.880724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T07:57:35.758686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.880592+0000","last_change":"2024-08-01T04:13:54.527536+0000","last_active":"2024-08-01T04:16:15.880592+0000","last_peered":"2024-08-01T04:16:15.880592+0000","last_clean":"2024-08-01T04:16:15.880592+0000","last_became_active":"2024-08-01T04:13:54.526848+0000","last_became_peered":"2024-08-01T04:13:54.526848+0000","last_unstale":"2024-08-01T04:16:15.880592+0000","last_undegraded":"2024-08-01T04:16:15.880592+0000","last_fullsized":"2024-08-01T04:16:15.880592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:53.458584+0000","last_clean_scrub_stamp":"2024-08-01T04:13:53.458584+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:15:34.085132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-08-01T04:16:15.841053+0000","last_change":"2024-08-01T04:13:51.469039+0000","last_active":"2024-08-01T04:16:15.841053+0000","last_peered":"2024-08-01T04:16:15.841053+0000","last_clean":"2024-08-01T04:16:15.841053+0000","last_became_active":"2024-08-01T04:13:51.468600+0000","last_became_peered":"2024-08-01T04:13:51.468600+0000","last_unstale":"2024-08-01T04:16:15.841053+0000","last_undegraded":"2024-08-01T04:16:15.841053+0000","last_fullsized":"2024-08-01T04:16:15.841053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-01T04:13:50.436308+0000","last_clean_scrub_stamp":"2024-08-01T04:13:50.436308+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-02T06:45:42.138891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799243,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1270,"num_read_kb":855,"num_write":428,"num_write_kb":2362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":2397729,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107622,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449133,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":493767,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678067,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32356,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712796,"statfs":{"total":95995035648,"available":95961903104,"internally_reserved":0,"allocated":1200128,"data_stored":1486354,"data_compressed":65569,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":887219,"data_compressed":33782,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":887139,"data_compressed":33782,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1088084,"data_compressed":50367,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495686,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":493392,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":1414689,"data_compressed":65569,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":495140,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":1215151,"data_compressed":48984,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":567409,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":564748,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":494663,"data_compressed":14493,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":331019,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":402684,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-01T04:16:39.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:16:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 -- 172.21.15.3:0/3092459682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 msgr2=0x7ff23c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:40.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 --2- 172.21.15.3:0/3092459682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c107ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009cd0 tx=0x7ff22c02a8f0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 -- 172.21.15.3:0/3092459682 shutdown_connections 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 --2- 172.21.15.3:0/3092459682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 --2- 172.21.15.3:0/3092459682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23c1016f0 0x7ff23c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 --2- 172.21.15.3:0/3092459682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c107ad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 -- 172.21.15.3:0/3092459682 >> 172.21.15.3:0/3092459682 conn(0x7ff23c0fd470 msgr2=0x7ff23c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.670+0000 7ff23b200640 1 -- 172.21.15.3:0/3092459682 shutdown_connections 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.671+0000 7ff23b200640 1 -- 172.21.15.3:0/3092459682 wait complete. 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 Processor -- start 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 -- start start 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23c1016f0 0x7ff23c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:40.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c111b40 con 0x7ff23c1076f0 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1119c0 con 0x7ff23c102090 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.672+0000 7ff23b200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c111cc0 con 0x7ff23c1016f0 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff23a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:37432/0 (socket says 172.21.15.3:37432) 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 -- 172.21.15.3:0/2894630616 learned_addr learned my addr 172.21.15.3:0/2894630616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff23a800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46080/0 (socket says 172.21.15.3:46080) 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23c1016f0 0x7ff23c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:40.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 -- 172.21.15.3:0/2894630616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23c1016f0 msgr2=0x7ff23c19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23c1016f0 0x7ff23c19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 -- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 msgr2=0x7ff23c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff239400640 1 -- 172.21.15.3:0/2894630616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c19f1d0 con 0x7ff23c102090 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.673+0000 7ff23a800640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.674+0000 7ff239400640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c19a760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff22400d770 tx=0x7ff22400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.674+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224017070 con 0x7ff23c102090 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.674+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c19f4c0 con 0x7ff23c102090 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.674+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff224005500 con 0x7ff23c102090 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.674+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224004e00 con 0x7ff23c102090 2024-08-01T04:16:40.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.674+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1a6da0 con 0x7ff23c102090 2024-08-01T04:16:40.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.675+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c101b50 con 0x7ff23c102090 2024-08-01T04:16:40.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.677+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7ff224004fa0 con 0x7ff23c102090 2024-08-01T04:16:40.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.679+0000 7ff232a00640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff2200a2e10 0x7ff2200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.680+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff2240cf190 con 0x7ff23c102090 2024-08-01T04:16:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.680+0000 7ff239e00640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff2200a2e10 0x7ff2200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.680+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff22409b470 con 0x7ff23c102090 2024-08-01T04:16:40.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.681+0000 7ff239e00640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff2200a2e10 0x7ff2200a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff22c004af0 tx=0x7ff22c009820 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:40.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.912+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff23c1a7000 con 0x7ff23c102090 2024-08-01T04:16:40.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.913+0000 7ff232a00640 1 -- 172.21.15.3:0/2894630616 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7ff22409f770 con 0x7ff23c102090 2024-08-01T04:16:40.915 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:40.916 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:16:40.916 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:16:40.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff2200a2e10 msgr2=0x7ff2200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff2200a2e10 0x7ff2200a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff22c004af0 tx=0x7ff22c009820 comp rx=0 tx=0).stop 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 msgr2=0x7ff23c19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c19a760 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff22400d770 tx=0x7ff22400dc30 comp rx=0 tx=0).stop 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 shutdown_connections 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff23c1076f0 0x7ff23c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff23c102090 0x7ff23c19a760 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff2200a2e10 0x7ff2200a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 --2- 172.21.15.3:0/2894630616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23c1016f0 0x7ff23c19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 >> 172.21.15.3:0/2894630616 conn(0x7ff23c0fd470 msgr2=0x7ff23c10cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 shutdown_connections 2024-08-01T04:16:40.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:40.916+0000 7ff23b200640 1 -- 172.21.15.3:0/2894630616 wait complete. 2024-08-01T04:16:40.937 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:40 smithi003 ceph-mon[32262]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-01T04:16:40.937 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:40 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2894630616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:41.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:40 smithi163 ceph-mon[37383]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-01T04:16:41.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:40 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2894630616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:40 smithi160 ceph-mon[37224]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-01T04:16:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:40 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2894630616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:41.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:16:42.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:41 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:16:42.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:41 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:16:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:42.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:41 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 -- 172.21.15.3:0/489349290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441007b0 msgr2=0x7fe544100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 --2- 172.21.15.3:0/489349290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441007b0 0x7fe544100c10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009930 tx=0x7fe52c02a420 comp rx=0 tx=0).stop 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 -- 172.21.15.3:0/489349290 shutdown_connections 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 --2- 172.21.15.3:0/489349290 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe544101150 0x7fe54410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 --2- 172.21.15.3:0/489349290 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441007b0 0x7fe544100c10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 --2- 172.21.15.3:0/489349290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5441067b0 0x7fe544106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 -- 172.21.15.3:0/489349290 >> 172.21.15.3:0/489349290 conn(0x7fe5440fc4d0 msgr2=0x7fe5440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:43.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 -- 172.21.15.3:0/489349290 shutdown_connections 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.056+0000 7fe549a00640 1 -- 172.21.15.3:0/489349290 wait complete. 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.057+0000 7fe549a00640 1 Processor -- start 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.057+0000 7fe549a00640 1 -- start start 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe549a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 0x7fe54419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe549a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe544101150 0x7fe54419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe549a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441067b0 0x7fe54419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe549a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe544110c00 con 0x7fe5441067b0 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe549a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe544110a80 con 0x7fe5441007b0 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe549a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe544110d80 con 0x7fe544101150 2024-08-01T04:16:43.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe542a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 0x7fe54419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe542a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 0x7fe54419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:54174/0 (socket says 172.21.15.3:54174) 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe542a00640 1 -- 172.21.15.3:0/243721280 learned_addr learned my addr 172.21.15.3:0/243721280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.058+0000 7fe53a000640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe544101150 0x7fe54419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe543400640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441067b0 0x7fe54419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe542a00640 1 -- 172.21.15.3:0/243721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe544101150 msgr2=0x7fe54419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe542a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe544101150 0x7fe54419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe542a00640 1 -- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441067b0 msgr2=0x7fe54419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe542a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441067b0 0x7fe54419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe542a00640 1 -- 172.21.15.3:0/243721280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54419f350 con 0x7fe5441007b0 2024-08-01T04:16:43.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe543400640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441067b0 0x7fe54419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe542a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 0x7fe54419a3a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9f0 tx=0x7fe53400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.059+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53400cd80 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.060+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe534005500 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.060+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54419f640 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.060+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534010690 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.060+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5441a6f20 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.061+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe544100c10 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.063+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7fe5340046c0 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.064+0000 7fe53be00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5280a2e10 0x7fe5280a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.064+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe5340cfc00 con 0x7fe5441007b0 2024-08-01T04:16:43.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.064+0000 7fe53a000640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5280a2e10 0x7fe5280a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.065+0000 7fe53a000640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5280a2e10 0x7fe5280a52d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe52c006010 tx=0x7fe52c000bc0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:43.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.068+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe53409c530 con 0x7fe5441007b0 2024-08-01T04:16:43.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:42 smithi163 ceph-mon[37383]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-08-01T04:16:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:42 smithi003 ceph-mon[32262]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-08-01T04:16:43.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.305+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5441a70f0 con 0x7fe5441007b0 2024-08-01T04:16:43.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.306+0000 7fe53be00640 1 -- 172.21.15.3:0/243721280 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7fe53409be30 con 0x7fe5441007b0 2024-08-01T04:16:43.307 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:43.308 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:16:43.308 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:16:43.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5280a2e10 msgr2=0x7fe5280a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:43.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5280a2e10 0x7fe5280a52d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe52c006010 tx=0x7fe52c000bc0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 msgr2=0x7fe54419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 0x7fe54419a3a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9f0 tx=0x7fe53400edf0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 shutdown_connections 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5441067b0 0x7fe54419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5280a2e10 0x7fe5280a52d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe544101150 0x7fe54419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 --2- 172.21.15.3:0/243721280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5441007b0 0x7fe54419a3a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.309+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 >> 172.21.15.3:0/243721280 conn(0x7fe5440fc4d0 msgr2=0x7fe54406a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.310+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 shutdown_connections 2024-08-01T04:16:43.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:43.310+0000 7fe549a00640 1 -- 172.21.15.3:0/243721280 wait complete. 2024-08-01T04:16:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:42 smithi160 ceph-mon[37224]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 37 op/s 2024-08-01T04:16:44.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:43 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/243721280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:44.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:16:44.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:43 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/243721280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:44.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:43 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/243721280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:44.695 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:45.111 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:44 smithi003 ceph-mon[32262]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 KiB/s wr, 34 op/s 2024-08-01T04:16:45.197 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:44 smithi163 ceph-mon[37383]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 KiB/s wr, 34 op/s 2024-08-01T04:16:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 -- 172.21.15.3:0/4239068550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 msgr2=0x7ff71c110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 --2- 172.21.15.3:0/4239068550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c110990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff714009cd0 tx=0x7ff71402a8f0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 -- 172.21.15.3:0/4239068550 shutdown_connections 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 --2- 172.21.15.3:0/4239068550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c110990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 --2- 172.21.15.3:0/4239068550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 0x7ff71c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 --2- 172.21.15.3:0/4239068550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff71c109ac0 0x7ff71c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 -- 172.21.15.3:0/4239068550 >> 172.21.15.3:0/4239068550 conn(0x7ff71c0ff7c0 msgr2=0x7ff71c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 -- 172.21.15.3:0/4239068550 shutdown_connections 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.368+0000 7ff720a00640 1 -- 172.21.15.3:0/4239068550 wait complete. 2024-08-01T04:16:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 Processor -- start 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 -- start start 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 0x7ff71c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff71c109ac0 0x7ff71c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c113f10 con 0x7ff71c103ac0 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c113d90 con 0x7ff71c109ac0 2024-08-01T04:16:45.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.370+0000 7ff720a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c114090 con 0x7ff71c104460 2024-08-01T04:16:45.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:33792/0 (socket says 172.21.15.3:33792) 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 -- 172.21.15.3:0/1040970682 learned_addr learned my addr 172.21.15.3:0/1040970682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719e00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 0x7ff71c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 -- 172.21.15.3:0/1040970682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff71c109ac0 msgr2=0x7ff71c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff71c109ac0 0x7ff71c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 -- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 msgr2=0x7ff71c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 0x7ff71c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719400640 1 -- 172.21.15.3:0/1040970682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c19f2e0 con 0x7ff71c104460 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.371+0000 7ff719e00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 0x7ff71c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.372+0000 7ff719400640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c19a870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff70800e9f0 tx=0x7ff70800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.372+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70800cd80 con 0x7ff71c104460 2024-08-01T04:16:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.372+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708010910 con 0x7ff71c104460 2024-08-01T04:16:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.372+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708018980 con 0x7ff71c104460 2024-08-01T04:16:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.372+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c19f5d0 con 0x7ff71c104460 2024-08-01T04:16:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.372+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a6eb0 con 0x7ff71c104460 2024-08-01T04:16:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.373+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c103f20 con 0x7ff71c104460 2024-08-01T04:16:45.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.377+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7ff708010ac0 con 0x7ff71c104460 2024-08-01T04:16:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.378+0000 7ff712a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7000a2e90 0x7ff7000a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.378+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff7080cfb80 con 0x7ff71c104460 2024-08-01T04:16:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.378+0000 7ff719e00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7000a2e90 0x7ff7000a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.379+0000 7ff719e00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7000a2e90 0x7ff7000a5350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005fd0 tx=0x7ff70c0094d0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.382+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff70809c4b0 con 0x7ff71c104460 2024-08-01T04:16:45.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:44 smithi160 ceph-mon[37224]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 37 KiB/s wr, 34 op/s 2024-08-01T04:16:45.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.584+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff71c1a7220 con 0x7ff71c104460 2024-08-01T04:16:45.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.586+0000 7ff712a00640 1 -- 172.21.15.3:0/1040970682 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23178 (secure 0 0 0) 0x7ff70809bdb0 con 0x7ff71c104460 2024-08-01T04:16:45.587 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:45.588 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":91,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:16:15.828445+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1348130449":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/346159818":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/4015366337":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2309230633":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7000a2e90 msgr2=0x7ff7000a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7000a2e90 0x7ff7000a5350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005fd0 tx=0x7ff70c0094d0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 msgr2=0x7ff71c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c19a870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff70800e9f0 tx=0x7ff70800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 shutdown_connections 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff71c109ac0 0x7ff71c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff7000a2e90 0x7ff7000a5350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff71c104460 0x7ff71c19a870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 --2- 172.21.15.3:0/1040970682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c103ac0 0x7ff71c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 >> 172.21.15.3:0/1040970682 conn(0x7ff71c0ff7c0 msgr2=0x7ff71c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 shutdown_connections 2024-08-01T04:16:45.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:45.589+0000 7ff720a00640 1 -- 172.21.15.3:0/1040970682 wait complete. 2024-08-01T04:16:46.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:45 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1040970682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:46.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:45 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1040970682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:46.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:45 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1040970682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:16:46.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-01T04:16:46.992 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:47.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:46 smithi163 ceph-mon[37383]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-01T04:16:47.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:46 smithi003 ceph-mon[32262]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-01T04:16:47.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:46 smithi160 ceph-mon[37224]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-01T04:16:48.718 INFO:teuthology.orchestra.run.smithi003.stdout:{"run_dir":"/var/run/ceph"} 2024-08-01T04:16:48.718 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-01T04:16:48.718 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 4805 2024-08-01T04:16:48.718 DEBUG:tasks.quiescer:Yielding 2024-08-01T04:16:48.719 INFO:teuthology.run_tasks:Running task workunit... 2024-08-01T04:16:48.733 INFO:tasks.workunit:Pulling workunits from ref d99c9769be1521395979e8c23d9073adbf1ff43f 2024-08-01T04:16:48.734 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-01T04:16:48.734 INFO:tasks.workunit:timeout=3h 2024-08-01T04:16:48.734 INFO:tasks.workunit:cleanup=True 2024-08-01T04:16:48.734 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:48.735 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:16:48.789 INFO:teuthology.orchestra.run.smithi003.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:48.789 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-01T04:16:48.789 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-08-01T04:16:48.790 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-01T04:16:48.790 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-01T04:16:48.790 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-08-01 04:14:22.196253375 +0000 2024-08-01T04:16:48.790 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-08-01 04:14:22.196253375 +0000 2024-08-01T04:16:48.790 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-08-01 04:16:16.119661983 +0000 2024-08-01T04:16:48.790 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-08-01 04:14:22.196253375 +0000 2024-08-01T04:16:48.790 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-01T04:16:48.790 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-01T04:16:48.919 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d99c9769be1521395979e8c23d9073adbf1ff43f 2024-08-01T04:16:48.977 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-01T04:16:49.210 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:49.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:48 smithi163 ceph-mon[37383]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:16:49.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:48 smithi003 ceph-mon[32262]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:16:49.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:48 smithi160 ceph-mon[37224]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:16:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 -- 172.21.15.3:0/2649318611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46101004b0 msgr2=0x7f4610100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 --2- 172.21.15.3:0/2649318611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46101004b0 0x7f4610100890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4600009900 tx=0x7f460002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:16:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 -- 172.21.15.3:0/2649318611 shutdown_connections 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 --2- 172.21.15.3:0/2649318611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f461010e060 0x7f4610110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 --2- 172.21.15.3:0/2649318611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4610100e60 0x7f461010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 --2- 172.21.15.3:0/2649318611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46101004b0 0x7f4610100890 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 -- 172.21.15.3:0/2649318611 >> 172.21.15.3:0/2649318611 conn(0x7f46100fc340 msgr2=0x7f46100fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 -- 172.21.15.3:0/2649318611 shutdown_connections 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.863+0000 7f460fe00640 1 -- 172.21.15.3:0/2649318611 wait complete. 2024-08-01T04:16:49.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.864+0000 7f460fe00640 1 Processor -- start 2024-08-01T04:16:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 -- start start 2024-08-01T04:16:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 0x7f4610102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4610100e60 0x7f4610102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:49.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f461010e060 0x7f4610103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113d60 con 0x7f46101004b0 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113be0 con 0x7f4610100e60 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113ee0 con 0x7f461010e060 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 0x7f4610102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4610100e60 0x7f4610102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 0x7f4610102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32916/0 (socket says 172.21.15.3:32916) 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.865+0000 7f460ea00640 1 -- 172.21.15.3:0/892508887 learned_addr learned my addr 172.21.15.3:0/892508887 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:49.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f461010e060 0x7f4610103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 -- 172.21.15.3:0/892508887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4610100e60 msgr2=0x7f4610102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4610100e60 0x7f4610102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 -- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 msgr2=0x7f4610102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 0x7f4610102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 -- 172.21.15.3:0/892508887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46101aa920 con 0x7f461010e060 2024-08-01T04:16:49.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460ea00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 0x7f4610102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.866+0000 7f460f400640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f461010e060 0x7f4610103210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4608009fd0 tx=0x7f460800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.867+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460800ce20 con 0x7f461010e060 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.867+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46101aac10 con 0x7f461010e060 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.867+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4608005020 con 0x7f461010e060 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.867+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080106d0 con 0x7f461010e060 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.867+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101ab150 con 0x7f461010e060 2024-08-01T04:16:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.868+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461006a520 con 0x7f461010e060 2024-08-01T04:16:49.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.870+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f46080051d0 con 0x7f461010e060 2024-08-01T04:16:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.873+0000 7f4614a00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f45f40a2e90 0x7f45f40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.873+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f46080cfdf0 con 0x7f461010e060 2024-08-01T04:16:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.873+0000 7f460ea00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f45f40a2e90 0x7f45f40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:49.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.874+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f460809c7b0 con 0x7f461010e060 2024-08-01T04:16:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:49.874+0000 7f460ea00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f45f40a2e90 0x7f45f40a5350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46000096c0 tx=0x7f46000046c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:50.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.113+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4610100890 con 0x7f461010e060 2024-08-01T04:16:50.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.114+0000 7f4614a00640 1 -- 172.21.15.3:0/892508887 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7f460809c0b0 con 0x7f461010e060 2024-08-01T04:16:50.115 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:16:50.115 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:16:50.115 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:16:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f45f40a2e90 msgr2=0x7f45f40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f45f40a2e90 0x7f45f40a5350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46000096c0 tx=0x7f46000046c0 comp rx=0 tx=0).stop 2024-08-01T04:16:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f461010e060 msgr2=0x7f4610103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:50.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f461010e060 0x7f4610103210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4608009fd0 tx=0x7f460800ef10 comp rx=0 tx=0).stop 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 shutdown_connections 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f45f40a2e90 0x7f45f40a5350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f461010e060 0x7f4610103210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4610100e60 0x7f4610102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 --2- 172.21.15.3:0/892508887 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46101004b0 0x7f4610102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 >> 172.21.15.3:0/892508887 conn(0x7f46100fc340 msgr2=0x7f46100fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 shutdown_connections 2024-08-01T04:16:50.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:50.117+0000 7f460fe00640 1 -- 172.21.15.3:0/892508887 wait complete. 2024-08-01T04:16:51.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-01T04:13:55.770404+0000', 'modified': '2024-08-01T04:16:17.179289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24548, 'mds_1': 24520, 'mds_2': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/3179303046', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 3179303046}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 3179303046}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/1698078935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 1698078935}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1698078935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.163:6839/4057809310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6838', 'nonce': 4057809310}, {'type': 'v1', 'addr': '172.21.15.163:6839', 'nonce': 4057809310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.160:6837/3127569351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 3127569351}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 3127569351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.3:6841/2435488646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 2435488646}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 2435488646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.160:6839/1741438883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 1741438883}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 1741438883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24548, 'qdb_cluster': [24506, 24520, 24548]} 2024-08-01T04:16:51.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-01T04:16:51.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.h status 2024-08-01T04:16:51.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:50 smithi163 ceph-mon[37383]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-01T04:16:51.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:50 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/892508887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:51.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:50 smithi003 ceph-mon[32262]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-01T04:16:51.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:50 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/892508887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:51.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:50 smithi160 ceph-mon[37224]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-01T04:16:51.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:50 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/892508887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:16:51.534 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 -- 172.21.15.3:0/3309189894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 msgr2=0x7f1e34076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 --2- 172.21.15.3:0/3309189894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e34076830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009900 tx=0x7f1e2802a410 comp rx=0 tx=0).stop 2024-08-01T04:16:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 -- 172.21.15.3:0/3309189894 shutdown_connections 2024-08-01T04:16:52.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 --2- 172.21.15.3:0/3309189894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e34112760 0x7f1e34114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 --2- 172.21.15.3:0/3309189894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e34076d70 0x7f1e34112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 --2- 172.21.15.3:0/3309189894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e34076830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 -- 172.21.15.3:0/3309189894 >> 172.21.15.3:0/3309189894 conn(0x7f1e340ff7c0 msgr2=0x7f1e34101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 -- 172.21.15.3:0/3309189894 shutdown_connections 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.209+0000 7f1e3b800640 1 -- 172.21.15.3:0/3309189894 wait complete. 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.210+0000 7f1e3b800640 1 Processor -- start 2024-08-01T04:16:52.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 -- start start 2024-08-01T04:16:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e341a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e34076d70 0x7f1e341a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e34112760 0x7f1e3419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34118320 con 0x7f1e34112760 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341181a0 con 0x7f1e34076450 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.211+0000 7f1e3b800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341184a0 con 0x7f1e34076d70 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e341a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e341a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45714/0 (socket says 172.21.15.3:45714) 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 -- 172.21.15.3:0/1559799028 learned_addr learned my addr 172.21.15.3:0/1559799028 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e39800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e34112760 0x7f1e3419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e33e00640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e34076d70 0x7f1e341a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 -- 172.21.15.3:0/1559799028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e34076d70 msgr2=0x7f1e341a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e34076d70 0x7f1e341a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 -- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e34112760 msgr2=0x7f1e3419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e34112760 0x7f1e3419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.212+0000 7f1e38e00640 1 -- 172.21.15.3:0/1559799028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3419ed10 con 0x7f1e34076450 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.213+0000 7f1e38e00640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e341a4420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e28004770 tx=0x7f1e28003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.213+0000 7f1e31600640 1 -- 172.21.15.3:0/1559799028 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2803f070 con 0x7f1e34076450 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.213+0000 7f1e31600640 1 -- 172.21.15.3:0/1559799028 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28031ca0 con 0x7f1e34076450 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.213+0000 7f1e31600640 1 -- 172.21.15.3:0/1559799028 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28035030 con 0x7f1e34076450 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.213+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3419efa0 con 0x7f1e34076450 2024-08-01T04:16:52.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.213+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e341ab2f0 con 0x7f1e34076450 2024-08-01T04:16:52.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.216+0000 7f1e31600640 1 -- 172.21.15.3:0/1559799028 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f1e28042070 con 0x7f1e34076450 2024-08-01T04:16:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 Processor -- start 2024-08-01T04:16:52.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 -- start start 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 0x55589d376dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d377310 0x55589d377750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d3820b0 0x55589d3844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55589d2dd940 con 0x55589d377310 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55589d2759a0 con 0x55589d378880 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.217+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55589d2d9900 con 0x55589d3820b0 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 0x55589d376dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 0x55589d376dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45726/0 (socket says 172.21.15.3:45726) 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 -- 172.21.15.3:0/1932269839 learned_addr learned my addr 172.21.15.3:0/1932269839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e33e00640 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d3820b0 0x55589d3844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e38e00640 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d377310 0x55589d377750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 -- 172.21.15.3:0/1932269839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d3820b0 msgr2=0x55589d3844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e31600640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1e1c0a2e90 0x7f1e1c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d3820b0 0x55589d3844a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 -- 172.21.15.3:0/1932269839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d377310 msgr2=0x55589d377750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d377310 0x55589d377750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e39800640 1 -- 172.21.15.3:0/1932269839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589d263ef0 con 0x55589d378880 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e33e00640 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d3820b0 0x55589d3844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.218+0000 7f1e31600640 1 -- 172.21.15.3:0/1559799028 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1e280efbe0 con 0x7f1e34076450 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.219+0000 7f1e39800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1e1c0a2e90 0x7f1e1c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.219+0000 7f1e39800640 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 0x55589d376dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009f30 tx=0x7f1e2c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b76893958b127c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.220+0000 7f1e26000640 1 -- 172.21.15.3:0/1932269839 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c019640 con 0x55589d378880 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.220+0000 7f1e26000640 1 -- 172.21.15.3:0/1932269839 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2c019cc0 con 0x55589d378880 2024-08-01T04:16:52.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.220+0000 7f1e26000640 1 -- 172.21.15.3:0/1932269839 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00fb90 con 0x55589d378880 2024-08-01T04:16:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.220+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/1932269839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 msgr2=0x55589d376dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.220+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 0x55589d376dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009f30 tx=0x7f1e2c0118d0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.220+0000 7f1e39800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1e1c0a2e90 0x7f1e1c0a5350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c020000 tx=0x7f1e2c021000 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/1932269839 shutdown_connections 2024-08-01T04:16:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d3820b0 0x55589d3844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d377310 0x55589d377750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/1932269839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d378880 0x55589d376dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/1932269839 >> 172.21.15.3:0/1932269839 conn(0x55589d33fa00 msgr2=0x55589d3739a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/1932269839 shutdown_connections 2024-08-01T04:16:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/1932269839 wait complete. 2024-08-01T04:16:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.221+0000 7f1e3e3bd740 1 -- start start 2024-08-01T04:16:52.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 Processor -- start 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d37f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d37fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55589d384db0 con 0x55589d378880 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55589d298a80 con 0x55589d3820b0 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55589d384f30 con 0x55589d377310 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.222+0000 7f1e3e3bd740 10 client.0 _ll_register_callbacks cb 0x55589d261140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e39800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d37fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d37f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e39800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d37fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45736/0 (socket says 172.21.15.3:45736) 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e39800640 1 -- 172.21.15.3:0/3766964080 learned_addr learned my addr 172.21.15.3:0/3766964080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 -- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 msgr2=0x55589d37fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d37fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 -- 172.21.15.3:0/3766964080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 msgr2=0x55589d37f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d37f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 -- 172.21.15.3:0/3766964080 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589d27c700 con 0x55589d377310 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e38e00640 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e280ed110 tx=0x7f1e2802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc1a7f9f2f352b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:52.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.223+0000 7f1e1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e1aa00640 1 -- 172.21.15.3:0/3766964080 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28049070 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e1aa00640 1 -- 172.21.15.3:0/3766964080 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1e2803b4f0 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e2c019cc0 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e1aa00640 1 -- 172.21.15.3:0/3766964080 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e280bc050 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e3e3bd740 10 client.25294 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55589d04f2b0 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e1aa00640 1 -- 172.21.15.3:0/3766964080 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2c019cc0 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e1aa00640 1 -- 172.21.15.3:0/3766964080 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1e280da080 con 0x55589d377310 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.224+0000 7f1e3e3bd740 10 client.25294 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.225+0000 7f1e3e3bd740 10 client.25294 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.225+0000 7f1e3e3bd740 20 client.25294 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.225+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d392f70 0x55589d3943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.225+0000 7f1e3e3bd740 4 client.25294 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:52.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.225+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589d2eaeb0 con 0x55589d392f70 2024-08-01T04:16:52.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.225+0000 7f1e33e00640 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d392f70 0x55589d3943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.226+0000 7f1e33e00640 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d392f70 0x55589d3943e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4737920605ca9762 server_cookie=d71e73cfc4145746 in_seq=0 out_seq=0 2024-08-01T04:16:52.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.226+0000 7f1e1aa00640 10 client.25294 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.226+0000 7f1e1aa00640 1 -- 172.21.15.3:0/3766964080 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55589d2eaeb0 con 0x55589d392f70 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.226+0000 7f1e1aa00640 10 client.25294 handle_command_reply: tid=0 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 client.25294 shutdown 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 msgr2=0x55589d37f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1e280ed110 tx=0x7f1e2802a410 comp rx=0 tx=0).stop 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 shutdown_connections 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d37fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d37f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d392f70 0x55589d3943e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/3766964080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 >> 172.21.15.3:0/3766964080 conn(0x55589d33fa00 msgr2=0x55589d3796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e1aa00640 0 client.25294 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 shutdown_connections 2024-08-01T04:16:52.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/3766964080 wait complete. 2024-08-01T04:16:52.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.227+0000 7f1e3e3bd740 20 client.25294 trim_cache size 0 max 16384 2024-08-01T04:16:52.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.247+0000 7f1e3e3bd740 1 Processor -- start 2024-08-01T04:16:52.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.247+0000 7f1e3e3bd740 1 -- start start 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d391c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d392160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55589d392720 con 0x55589d378880 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55589d392df0 con 0x55589d3820b0 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55589d3934c0 con 0x55589d377310 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e39800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d392160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:58882/0 (socket says 172.21.15.3:58882) 2024-08-01T04:16:52.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e38e00640 1 -- 172.21.15.3:0/592344208 learned_addr learned my addr 172.21.15.3:0/592344208 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:52.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.248+0000 7f1e33e00640 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d391c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e38e00640 1 -- 172.21.15.3:0/592344208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 msgr2=0x55589d392160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e38e00640 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d392160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e38e00640 1 -- 172.21.15.3:0/592344208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 msgr2=0x55589d391c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e38e00640 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d391c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e38e00640 1 -- 172.21.15.3:0/592344208 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589d381ae0 con 0x55589d377310 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e38e00640 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e2803b4a0 tx=0x7f1e280c4780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e7c628276709c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e25600640 1 -- 172.21.15.3:0/592344208 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e280ab070 con 0x55589d377310 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e25600640 1 -- 172.21.15.3:0/592344208 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e280d7050 con 0x55589d377310 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.249+0000 7f1e25600640 1 -- 172.21.15.3:0/592344208 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28049070 con 0x55589d377310 2024-08-01T04:16:52.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/592344208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 msgr2=0x55589d37f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e2803b4a0 tx=0x7f1e280c4780 comp rx=0 tx=0).stop 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/592344208 shutdown_connections 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d392160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d391c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/592344208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d37f900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/592344208 >> 172.21.15.3:0/592344208 conn(0x55589d33fa00 msgr2=0x55589d36e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/592344208 shutdown_connections 2024-08-01T04:16:52.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.250+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/592344208 wait complete. 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 -- start start 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 Processor -- start 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d395bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d396130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d3fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55589d394260 con 0x55589d378880 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55589d393b90 con 0x55589d3820b0 2024-08-01T04:16:52.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.251+0000 7f1e3e3bd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55589d394930 con 0x55589d377310 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e3e3bd740 10 client.0 _ll_register_callbacks cb 0x7f1e280f31a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d395bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d396130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e38e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d395bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:58898/0 (socket says 172.21.15.3:58898) 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d396130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39354/0 (socket says 172.21.15.3:39354) 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e38e00640 1 -- 172.21.15.3:0/2964808325 learned_addr learned my addr 172.21.15.3:0/2964808325 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e39800640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d3fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 -- 172.21.15.3:0/2964808325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 msgr2=0x55589d395bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d395bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 -- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 msgr2=0x55589d3fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d3fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e33e00640 1 -- 172.21.15.3:0/2964808325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589d3fbbb0 con 0x55589d378880 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.252+0000 7f1e38e00640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d395bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e33e00640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d396130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55589d380670 tx=0x7f1e20010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2be39fb82ab39bc server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:52.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e1aa00640 1 -- 172.21.15.3:0/2964808325 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20004230 con 0x55589d378880 2024-08-01T04:16:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e1aa00640 1 -- 172.21.15.3:0/2964808325 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1e200043d0 con 0x55589d378880 2024-08-01T04:16:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e280d7050 con 0x55589d378880 2024-08-01T04:16:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e1aa00640 1 -- 172.21.15.3:0/2964808325 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1e20005020 con 0x55589d378880 2024-08-01T04:16:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e3e3bd740 10 client.15180 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:52.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.253+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55589d3fce50 con 0x55589d378880 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e1aa00640 1 -- 172.21.15.3:0/2964808325 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e280d7050 con 0x55589d378880 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e1aa00640 1 -- 172.21.15.3:0/2964808325 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1e20025030 con 0x55589d378880 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e3e3bd740 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e3e3bd740 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e3e3bd740 20 client.15180 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d4019d0 0x55589d4006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e3e3bd740 4 client.15180 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "status"}) -- 0x55589d2e53c0 con 0x55589d4019d0 2024-08-01T04:16:52.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.254+0000 7f1e38e00640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d4019d0 0x55589d4006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e38e00640 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d4019d0 0x55589d4006e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a73ccfa6f5d93402 server_cookie=de71d26e0b98d41b in_seq=0 out_seq=0 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e1aa00640 10 client.15180 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e1aa00640 1 -- 172.21.15.3:0/2964808325 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55589d2e53c0 con 0x55589d4019d0 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e1aa00640 10 client.15180 handle_command_reply: tid=0 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e3e3bd740 1 client.15180 shutdown 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 msgr2=0x55589d396130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.255+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d396130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55589d380670 tx=0x7f1e20010800 comp rx=0 tx=0).stop 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 shutdown_connections 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55589d3820b0 0x55589d3fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55589d378880 0x55589d396130 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55589d4019d0 0x55589d4006e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 --2- 172.21.15.3:0/2964808325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55589d377310 0x55589d395bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 >> 172.21.15.3:0/2964808325 conn(0x55589d33fa00 msgr2=0x55589d391c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e1aa00640 0 client.15180 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 shutdown_connections 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 1 -- 172.21.15.3:0/2964808325 wait complete. 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.256+0000 7f1e3e3bd740 20 client.15180 trim_cache size 0 max 16384 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24506, 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:16:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 164.66099058099999, 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 26, 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 91, 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 204.134679941 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1e1c0a2e90 msgr2=0x7f1e1c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1e1c0a2e90 0x7f1e1c0a5350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c020000 tx=0x7f1e2c021000 comp rx=0 tx=0).stop 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 msgr2=0x7f1e341a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e341a4420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e28004770 tx=0x7f1e28003c10 comp rx=0 tx=0).stop 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 shutdown_connections 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e34112760 0x7f1e3419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1e1c0a2e90 0x7f1e1c0a5350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e34076d70 0x7f1e341a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 --2- 172.21.15.3:0/1559799028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e34076450 0x7f1e341a4420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 >> 172.21.15.3:0/1559799028 conn(0x7f1e340ff7c0 msgr2=0x7f1e34101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 shutdown_connections 2024-08-01T04:16:52.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:52.257+0000 7f1e3b800640 1 -- 172.21.15.3:0/1559799028 wait complete. 2024-08-01T04:16:53.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:52 smithi003 ceph-mon[32262]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:16:53.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.a status 2024-08-01T04:16:53.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:52 smithi163 ceph-mon[37383]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:16:53.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:52 smithi160 ceph-mon[37224]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:16:53.666 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 -- 172.21.15.3:0/3588349099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c107d50 msgr2=0x7eff3c108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 --2- 172.21.15.3:0/3588349099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c107d50 0x7eff3c108170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff24009930 tx=0x7eff2402a420 comp rx=0 tx=0).stop 2024-08-01T04:16:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 -- 172.21.15.3:0/3588349099 shutdown_connections 2024-08-01T04:16:54.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 --2- 172.21.15.3:0/3588349099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff3c1086b0 0x7eff3c114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 --2- 172.21.15.3:0/3588349099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c107d50 0x7eff3c108170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 --2- 172.21.15.3:0/3588349099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff3c10dd90 0x7eff3c10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.324+0000 7eff3be00640 1 -- 172.21.15.3:0/3588349099 >> 172.21.15.3:0/3588349099 conn(0x7eff3c0776b0 msgr2=0x7eff3c079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.325+0000 7eff3be00640 1 -- 172.21.15.3:0/3588349099 shutdown_connections 2024-08-01T04:16:54.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.325+0000 7eff3be00640 1 -- 172.21.15.3:0/3588349099 wait complete. 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.326+0000 7eff3be00640 1 Processor -- start 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.326+0000 7eff3be00640 1 -- start start 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.326+0000 7eff3be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff3c107d50 0x7eff3c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.326+0000 7eff3be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff3c1086b0 0x7eff3c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 0x7eff3c1a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c1180e0 con 0x7eff3c10dd90 2024-08-01T04:16:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3be00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c117f60 con 0x7eff3c107d50 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c118260 con 0x7eff3c1086b0 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 0x7eff3c1a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff3c1086b0 0x7eff3c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff3c107d50 0x7eff3c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 0x7eff3c1a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39382/0 (socket says 172.21.15.3:39382) 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 -- 172.21.15.3:0/3788316530 learned_addr learned my addr 172.21.15.3:0/3788316530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 -- 172.21.15.3:0/3788316530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff3c1086b0 msgr2=0x7eff3c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff3c1086b0 0x7eff3c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 -- 172.21.15.3:0/3788316530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff3c107d50 msgr2=0x7eff3c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff3c107d50 0x7eff3c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.327+0000 7eff3b400640 1 -- 172.21.15.3:0/3788316530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c1a35b0 con 0x7eff3c10dd90 2024-08-01T04:16:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.328+0000 7eff3b400640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 0x7eff3c1a2ed0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff3400ea60 tx=0x7eff3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.328+0000 7eff33400640 1 -- 172.21.15.3:0/3788316530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3400ce40 con 0x7eff3c10dd90 2024-08-01T04:16:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.328+0000 7eff33400640 1 -- 172.21.15.3:0/3788316530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff34005020 con 0x7eff3c10dd90 2024-08-01T04:16:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.328+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c1a38a0 con 0x7eff3c10dd90 2024-08-01T04:16:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.328+0000 7eff33400640 1 -- 172.21.15.3:0/3788316530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34010690 con 0x7eff3c10dd90 2024-08-01T04:16:54.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.328+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c1ab180 con 0x7eff3c10dd90 2024-08-01T04:16:54.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.333+0000 7eff33400640 1 -- 172.21.15.3:0/3788316530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7eff34005520 con 0x7eff3c10dd90 2024-08-01T04:16:54.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.333+0000 7eff445cc740 1 Processor -- start 2024-08-01T04:16:54.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.333+0000 7eff445cc740 1 -- start start 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff445cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff445cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 0x55f4308b9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff445cc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff445cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f43081f940 con 0x55f4308baa50 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff445cc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f4307b79a0 con 0x55f4308c4280 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff445cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f43081b900 con 0x55f4308b94e0 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 0x55f4308b9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 0x55f4308b9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:58904/0 (socket says 172.21.15.3:58904) 2024-08-01T04:16:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff3aa00640 1 -- 172.21.15.3:0/2997910542 learned_addr learned my addr 172.21.15.3:0/2997910542 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:54.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.334+0000 7eff3a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff3aa00640 1 -- 172.21.15.3:0/2997910542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 msgr2=0x55f4308c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff3aa00640 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff3aa00640 1 -- 172.21.15.3:0/2997910542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 msgr2=0x55f4308b8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff3aa00640 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308b8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff3aa00640 1 -- 172.21.15.3:0/2997910542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4307a5ef0 con 0x55f4308b94e0 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff3aa00640 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 0x55f4308b9920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009c70 tx=0x7eff2c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b70fddae091222a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff2be00640 1 -- 172.21.15.3:0/2997910542 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c004230 con 0x55f4308b94e0 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.335+0000 7eff33400640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff200a2e10 0x7eff200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.336+0000 7eff2be00640 1 -- 172.21.15.3:0/2997910542 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c0043d0 con 0x55f4308b94e0 2024-08-01T04:16:54.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.336+0000 7eff2be00640 1 -- 172.21.15.3:0/2997910542 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c019740 con 0x55f4308b94e0 2024-08-01T04:16:54.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.336+0000 7eff33400640 1 -- 172.21.15.3:0/3788316530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7eff340cfd60 con 0x7eff3c10dd90 2024-08-01T04:16:54.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.336+0000 7eff3a000640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff200a2e10 0x7eff200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.336+0000 7eff445cc740 1 -- 172.21.15.3:0/2997910542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 msgr2=0x55f4308b9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.336+0000 7eff445cc740 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 0x55f4308b9920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009c70 tx=0x7eff2c011810 comp rx=0 tx=0).stop 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 -- 172.21.15.3:0/2997910542 shutdown_connections 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff3a000640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff200a2e10 0x7eff200a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff24006010 tx=0x7eff24005be0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308b94e0 0x55f4308b9920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 --2- 172.21.15.3:0/2997910542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 -- 172.21.15.3:0/2997910542 >> 172.21.15.3:0/2997910542 conn(0x55f430881a00 msgr2=0x55f4308b5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 -- 172.21.15.3:0/2997910542 shutdown_connections 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 -- 172.21.15.3:0/2997910542 wait complete. 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.337+0000 7eff445cc740 1 -- start start 2024-08-01T04:16:54.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 Processor -- start 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308c4280 0x55f4308c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f4308c6f80 con 0x55f4308c4280 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f4307daa80 con 0x55f4308b94e0 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f4308c7100 con 0x55f4308baa50 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.338+0000 7eff445cc740 10 client.0 _ll_register_callbacks cb 0x55f4307a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:54.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308c11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45806/0 (socket says 172.21.15.3:45806) 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 -- 172.21.15.3:0/305987497 learned_addr learned my addr 172.21.15.3:0/305987497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3a000640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3b400640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308c4280 0x55f4308c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 -- 172.21.15.3:0/305987497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 msgr2=0x55f4308c16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308c16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 -- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308c4280 msgr2=0x55f4308c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308c4280 0x55f4308c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 -- 172.21.15.3:0/305987497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4307be700 con 0x55f4308b94e0 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff3aa00640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308c11b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff2c005bc0 tx=0x7eff2c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27e8164b4ca886b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.339+0000 7eff28c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:16:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.340+0000 7eff28c00640 1 -- 172.21.15.3:0/305987497 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c027070 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.340+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2c0043d0 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.340+0000 7eff28c00640 1 -- 172.21.15.3:0/305987497 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7eff2c003800 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.340+0000 7eff28c00640 1 -- 172.21.15.3:0/305987497 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff2c033050 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.340+0000 7eff445cc740 10 client.25328 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.340+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4305912b0 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff28c00640 1 -- 172.21.15.3:0/305987497 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c0043d0 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff28c00640 1 -- 172.21.15.3:0/305987497 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7eff2c018070 con 0x55f4308b94e0 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff445cc740 10 client.25328 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff445cc740 10 client.25328 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff445cc740 20 client.25328 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:54.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff445cc740 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4308d4d90 0x55f4308d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff445cc740 4 client.25328 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:54.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.341+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43082ceb0 con 0x55f4308d4d90 2024-08-01T04:16:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.342+0000 7eff3a000640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4308d4d90 0x55f4308d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.342+0000 7eff3a000640 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4308d4d90 0x55f4308d6200 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5611cc49fcb04a46 server_cookie=a5563d05ec2a6c85 in_seq=0 out_seq=0 2024-08-01T04:16:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.342+0000 7eff28c00640 10 client.25328 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff28c00640 1 -- 172.21.15.3:0/305987497 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f43082ceb0 con 0x55f4308d4d90 2024-08-01T04:16:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff28c00640 10 client.25328 handle_command_reply: tid=0 2024-08-01T04:16:54.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 client.25328 shutdown 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 msgr2=0x55f4308c11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308c11b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7eff2c005bc0 tx=0x7eff2c009ee0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 shutdown_connections 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4308d4d90 0x55f4308d6200 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308c4280 0x55f4308c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 --2- 172.21.15.3:0/305987497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308c11b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff28c00640 0 client.25328 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.343+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 >> 172.21.15.3:0/305987497 conn(0x55f430881a00 msgr2=0x55f4308bb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.344+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 shutdown_connections 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.344+0000 7eff445cc740 1 -- 172.21.15.3:0/305987497 wait complete. 2024-08-01T04:16:54.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.344+0000 7eff445cc740 20 client.25328 trim_cache size 0 max 16384 2024-08-01T04:16:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 Processor -- start 2024-08-01T04:16:54.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 -- start start 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308c4280 0x55f4308c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f4308d4540 con 0x55f4308baa50 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f4308d4c10 con 0x55f4308b94e0 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.364+0000 7eff445cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f4308d52e0 con 0x55f4308c4280 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45820/0 (socket says 172.21.15.3:45820) 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 -- 172.21.15.3:0/3227091206 learned_addr learned my addr 172.21.15.3:0/3227091206 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:54.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3b400640 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308c4280 0x55f4308c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 -- 172.21.15.3:0/3227091206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308c4280 msgr2=0x55f4308c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308c4280 0x55f4308c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 -- 172.21.15.3:0/3227091206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 msgr2=0x55f4308c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 -- 172.21.15.3:0/3227091206 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4308ba690 con 0x55f4308b94e0 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3b400640 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308c4280 0x55f4308c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.365+0000 7eff3aa00640 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308b5d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7eff2c01a4c0 tx=0x7eff2c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470f769bf42b6c95 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff2b400640 1 -- 172.21.15.3:0/3227091206 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c027030 con 0x55f4308b94e0 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff2b400640 1 -- 172.21.15.3:0/3227091206 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c015070 con 0x55f4308b94e0 2024-08-01T04:16:54.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff2b400640 1 -- 172.21.15.3:0/3227091206 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c005640 con 0x55f4308b94e0 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 -- 172.21.15.3:0/3227091206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 msgr2=0x55f4308b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308b5d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7eff2c01a4c0 tx=0x7eff2c001e50 comp rx=0 tx=0).stop 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 -- 172.21.15.3:0/3227091206 shutdown_connections 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308c4280 0x55f4308c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308baa50 0x55f4308c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 --2- 172.21.15.3:0/3227091206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308b94e0 0x55f4308b5d20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 -- 172.21.15.3:0/3227091206 >> 172.21.15.3:0/3227091206 conn(0x55f430881a00 msgr2=0x55f4308b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.366+0000 7eff445cc740 1 -- 172.21.15.3:0/3227091206 shutdown_connections 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.367+0000 7eff445cc740 1 -- 172.21.15.3:0/3227091206 wait complete. 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.367+0000 7eff445cc740 1 -- start start 2024-08-01T04:16:54.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.367+0000 7eff445cc740 1 Processor -- start 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 0x55f4308bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308bd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f4308d6080 con 0x55f4308b94e0 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f4308d59b0 con 0x55f4308c4280 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f4308d6750 con 0x55f4308baa50 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff445cc740 10 client.0 _ll_register_callbacks cb 0x7eff340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 0x55f4308bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308bd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 0x55f4308bd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39424/0 (socket says 172.21.15.3:39424) 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308bd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:58952/0 (socket says 172.21.15.3:58952) 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308bdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45836/0 (socket says 172.21.15.3:45836) 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 -- 172.21.15.3:0/2471173439 learned_addr learned my addr 172.21.15.3:0/2471173439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 -- 172.21.15.3:0/2471173439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 msgr2=0x55f4308bdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308bdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 -- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 msgr2=0x55f4308bd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.368+0000 7eff3a000640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 0x55f4308bd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff3a000640 1 -- 172.21.15.3:0/2471173439 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4308be480 con 0x55f4308baa50 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff3b400640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff3aa00640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 0x55f4308bd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff3a000640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308bd980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff2402aa10 tx=0x7eff24009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c5e6b77845561a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:54.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff28c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff28c00640 1 -- 172.21.15.3:0/2471173439 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24035050 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2c015070 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.369+0000 7eff28c00640 1 -- 172.21.15.3:0/2471173439 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7eff240048c0 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff28c00640 1 -- 172.21.15.3:0/2471173439 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff24004ac0 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff445cc740 10 client.25318 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f43093f820 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff28c00640 1 -- 172.21.15.3:0/2471173439 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c015070 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff28c00640 1 -- 172.21.15.3:0/2471173439 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7eff240035f0 con 0x55f4308baa50 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff445cc740 10 client.25318 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff445cc740 10 client.25318 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff445cc740 20 client.25318 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:54.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.370+0000 7eff445cc740 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4309443b0 0x55f4309430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:54.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.371+0000 7eff445cc740 4 client.25318 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:16:54.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.371+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "status"}) -- 0x55f4308273c0 con 0x55f4309443b0 2024-08-01T04:16:54.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.371+0000 7eff3aa00640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4309443b0 0x55f4309430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.371+0000 7eff3aa00640 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4309443b0 0x55f4309430c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c16c9a967c9be9d server_cookie=f63063ca6d1146b3 in_seq=0 out_seq=0 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.371+0000 7eff28c00640 10 client.25318 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff28c00640 1 -- 172.21.15.3:0/2471173439 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f4308273c0 con 0x55f4309443b0 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff28c00640 10 client.25318 handle_command_reply: tid=0 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 client.25318 shutdown 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 msgr2=0x55f4308bd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308bd980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff2402aa10 tx=0x7eff24009260 comp rx=0 tx=0).stop 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 shutdown_connections 2024-08-01T04:16:54.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f4309443b0 0x55f4309430c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f4308c4280 0x55f4308bdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f4308baa50 0x55f4308bd980 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 --2- 172.21.15.3:0/2471173439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f4308b94e0 0x55f4308bd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff28c00640 0 client.25318 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 >> 172.21.15.3:0/2471173439 conn(0x55f430881a00 msgr2=0x55f4308cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 shutdown_connections 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.372+0000 7eff445cc740 1 -- 172.21.15.3:0/2471173439 wait complete. 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff445cc740 20 client.25318 trim_cache size 0 max 16384 2024-08-01T04:16:54.377 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24520, 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 167.78751229900001, 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 26, 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 91, 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 195.69261129899999 2024-08-01T04:16:54.378 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff200a2e10 msgr2=0x7eff200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff200a2e10 0x7eff200a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff24006010 tx=0x7eff24005be0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 msgr2=0x7eff3c1a2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 0x7eff3c1a2ed0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff3400ea60 tx=0x7eff3400ef20 comp rx=0 tx=0).stop 2024-08-01T04:16:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 shutdown_connections 2024-08-01T04:16:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7eff3c10dd90 0x7eff3c1a2ed0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7eff200a2e10 0x7eff200a52d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff3c1086b0 0x7eff3c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 --2- 172.21.15.3:0/3788316530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff3c107d50 0x7eff3c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.373+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 >> 172.21.15.3:0/3788316530 conn(0x7eff3c0776b0 msgr2=0x7eff3c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.374+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 shutdown_connections 2024-08-01T04:16:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:54.374+0000 7eff3be00640 1 -- 172.21.15.3:0/3788316530 wait complete. 2024-08-01T04:16:55.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:54 smithi163 ceph-mon[37383]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:16:55.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:54 smithi003 ceph-mon[32262]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:16:55.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:54 smithi160 ceph-mon[37224]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:16:55.477 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.b status 2024-08-01T04:16:55.962 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:16:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 -- 172.21.15.3:0/258326108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d441086b0 msgr2=0x7f1d44114bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 --2- 172.21.15.3:0/258326108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d441086b0 0x7f1d44114bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009c10 tx=0x7f1d3c02a6f0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 -- 172.21.15.3:0/258326108 shutdown_connections 2024-08-01T04:16:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 --2- 172.21.15.3:0/258326108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d441086b0 0x7f1d44114bd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 --2- 172.21.15.3:0/258326108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d44107d50 0x7f1d44108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 --2- 172.21.15.3:0/258326108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d4410dd90 0x7f1d4410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 -- 172.21.15.3:0/258326108 >> 172.21.15.3:0/258326108 conn(0x7f1d440776b0 msgr2=0x7f1d44079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:56.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 -- 172.21.15.3:0/258326108 shutdown_connections 2024-08-01T04:16:56.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.636+0000 7f1d43e00640 1 -- 172.21.15.3:0/258326108 wait complete. 2024-08-01T04:16:56.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.637+0000 7f1d43e00640 1 Processor -- start 2024-08-01T04:16:56.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.637+0000 7f1d43e00640 1 -- start start 2024-08-01T04:16:56.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d44107d50 0x7f1d4419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 0x7f1d4419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d4410dd90 0x7f1d441a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441180a0 con 0x7f1d441086b0 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44117f20 con 0x7f1d4410dd90 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44118220 con 0x7f1d44107d50 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 0x7f1d4419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 0x7f1d4419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39436/0 (socket says 172.21.15.3:39436) 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 -- 172.21.15.3:0/3244151439 learned_addr learned my addr 172.21.15.3:0/3244151439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42a00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d44107d50 0x7f1d4419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d43400640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d4410dd90 0x7f1d441a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 -- 172.21.15.3:0/3244151439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d44107d50 msgr2=0x7f1d4419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d44107d50 0x7f1d4419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 -- 172.21.15.3:0/3244151439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d4410dd90 msgr2=0x7f1d441a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d4410dd90 0x7f1d441a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.638+0000 7f1d42000640 1 -- 172.21.15.3:0/3244151439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d441a3500 con 0x7f1d441086b0 2024-08-01T04:16:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.639+0000 7f1d42000640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 0x7f1d4419ea90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00e9f0 tx=0x7f1d2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.639+0000 7f1d3b400640 1 -- 172.21.15.3:0/3244151439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00ccc0 con 0x7f1d441086b0 2024-08-01T04:16:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.639+0000 7f1d3b400640 1 -- 172.21.15.3:0/3244151439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2c010910 con 0x7f1d441086b0 2024-08-01T04:16:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.639+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441a37f0 con 0x7f1d441086b0 2024-08-01T04:16:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.639+0000 7f1d3b400640 1 -- 172.21.15.3:0/3244151439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0188a0 con 0x7f1d441086b0 2024-08-01T04:16:56.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.639+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441ab0d0 con 0x7f1d441086b0 2024-08-01T04:16:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d3b400640 1 -- 172.21.15.3:0/3244151439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 138059+0+0 (secure 0 0 0) 0x7f1d2c010ac0 con 0x7f1d441086b0 2024-08-01T04:16:56.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 Processor -- start 2024-08-01T04:16:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 -- start start 2024-08-01T04:16:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 0x563fc8da5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8da6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8db3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563fc8d0c940 con 0x563fc8db1280 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563fc8ca49a0 con 0x563fc8da64e0 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.643+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563fc8d08900 con 0x563fc8da7a50 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 0x563fc8da5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d42000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8db3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 0x563fc8da5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:58990/0 (socket says 172.21.15.3:58990) 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 -- 172.21.15.3:0/4148810045 learned_addr learned my addr 172.21.15.3:0/4148810045 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:56.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d42a00640 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8da6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 -- 172.21.15.3:0/4148810045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 msgr2=0x563fc8da6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8da6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 -- 172.21.15.3:0/4148810045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 msgr2=0x563fc8db3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8db3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d43400640 1 -- 172.21.15.3:0/4148810045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc8c92ef0 con 0x563fc8da7a50 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d42a00640 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8da6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.644+0000 7f1d3b400640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d280a2e10 0x7f1d280a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.645+0000 7f1d3b400640 1 -- 172.21.15.3:0/3244151439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1d2c0cfc40 con 0x7f1d441086b0 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.645+0000 7f1d43400640 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 0x563fc8da5fa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009c10 tx=0x7f1d3c030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825074cfd70a43b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.645+0000 7f1d33e00640 1 -- 172.21.15.3:0/4148810045 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c041030 con 0x563fc8da7a50 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.645+0000 7f1d42a00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d280a2e10 0x7f1d280a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.645+0000 7f1d33e00640 1 -- 172.21.15.3:0/4148810045 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c037040 con 0x563fc8da7a50 2024-08-01T04:16:56.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.645+0000 7f1d33e00640 1 -- 172.21.15.3:0/4148810045 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c036040 con 0x563fc8da7a50 2024-08-01T04:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/4148810045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 msgr2=0x563fc8da5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d42a00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d280a2e10 0x7f1d280a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x563fc8da39d0 tx=0x7f1d34009260 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 0x563fc8da5fa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009c10 tx=0x7f1d3c030a30 comp rx=0 tx=0).stop 2024-08-01T04:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/4148810045 shutdown_connections 2024-08-01T04:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8db3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8da6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/4148810045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da7a50 0x563fc8da5fa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/4148810045 >> 172.21.15.3:0/4148810045 conn(0x563fc8d6ea00 msgr2=0x563fc8da2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/4148810045 shutdown_connections 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.646+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/4148810045 wait complete. 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.647+0000 7f1d4c2c0740 1 -- start start 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.647+0000 7f1d4c2c0740 1 Processor -- start 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da64e0 0x563fc8dae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8dae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8daec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563fc8db3f20 con 0x563fc8da64e0 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563fc8cc7a80 con 0x563fc8da7a50 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563fc8db40a0 con 0x563fc8db1280 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d4c2c0740 10 client.0 _ll_register_callbacks cb 0x563fc8c90140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d42a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da64e0 0x563fc8dae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d43400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8daec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d42a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da64e0 0x563fc8dae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39450/0 (socket says 172.21.15.3:39450) 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d42a00640 1 -- 172.21.15.3:0/1574905948 learned_addr learned my addr 172.21.15.3:0/1574905948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:56.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.648+0000 7f1d42000640 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8dae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d43400640 1 -- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 msgr2=0x563fc8dae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d43400640 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8dae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d43400640 1 -- 172.21.15.3:0/1574905948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da64e0 msgr2=0x563fc8dae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d43400640 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da64e0 0x563fc8dae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d43400640 1 -- 172.21.15.3:0/1574905948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc8cab700 con 0x563fc8db1280 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d43400640 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8daec70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005cc0 tx=0x7f1d3c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd186e2aa99f891 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:56.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d30c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d30c00640 1 -- 172.21.15.3:0/1574905948 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c04a070 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d30c00640 1 -- 172.21.15.3:0/1574905948 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1d3c031640 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.649+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c037040 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.650+0000 7f1d30c00640 1 -- 172.21.15.3:0/1574905948 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c056050 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.650+0000 7f1d4c2c0740 10 client.25330 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.650+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fc8a7e2b0 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.650+0000 7f1d30c00640 1 -- 172.21.15.3:0/1574905948 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c037040 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.650+0000 7f1d30c00640 1 -- 172.21.15.3:0/1574905948 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1d3c03c040 con 0x563fc8db1280 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.650+0000 7f1d4c2c0740 10 client.25330 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.651+0000 7f1d4c2c0740 10 client.25330 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-01T04:16:56.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.651+0000 7f1d4c2c0740 20 client.25330 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.651+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8dc1bc0 0x563fc8dc3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.651+0000 7f1d4c2c0740 4 client.25330 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:16:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.651+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc8d19eb0 con 0x563fc8dc1bc0 2024-08-01T04:16:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.651+0000 7f1d42a00640 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8dc1bc0 0x563fc8dc3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.652+0000 7f1d42a00640 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8dc1bc0 0x563fc8dc3030 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27ab35ac142e21b server_cookie=c6c4c35124d644ca in_seq=0 out_seq=0 2024-08-01T04:16:56.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.652+0000 7f1d30c00640 10 client.25330 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d30c00640 1 -- 172.21.15.3:0/1574905948 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563fc8d19eb0 con 0x563fc8dc1bc0 2024-08-01T04:16:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d30c00640 10 client.25330 handle_command_reply: tid=0 2024-08-01T04:16:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 client.25330 shutdown 2024-08-01T04:16:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 msgr2=0x563fc8daec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8daec70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005cc0 tx=0x7f1d3c009500 comp rx=0 tx=0).stop 2024-08-01T04:16:56.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 shutdown_connections 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8dc1bc0 0x563fc8dc3030 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8daec70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8dae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/1574905948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da64e0 0x563fc8dae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 >> 172.21.15.3:0/1574905948 conn(0x563fc8d6ea00 msgr2=0x563fc8da8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d30c00640 0 client.25330 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 shutdown_connections 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.653+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/1574905948 wait complete. 2024-08-01T04:16:56.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.654+0000 7f1d4c2c0740 20 client.25330 trim_cache size 0 max 16384 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.673+0000 7f1d4c2c0740 1 Processor -- start 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.673+0000 7f1d4c2c0740 1 -- start start 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da64e0 0x563fc8dc14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8daeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8da24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563fc8dc1370 con 0x563fc8db1280 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563fc8dc1a40 con 0x563fc8da7a50 2024-08-01T04:16:56.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563fc8dc2110 con 0x563fc8da64e0 2024-08-01T04:16:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d43400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8da24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d43400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8da24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39452/0 (socket says 172.21.15.3:39452) 2024-08-01T04:16:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d43400640 1 -- 172.21.15.3:0/232883602 learned_addr learned my addr 172.21.15.3:0/232883602 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d42a00640 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da64e0 0x563fc8dc14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.674+0000 7f1d42000640 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8daeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d42a00640 1 -- 172.21.15.3:0/232883602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 msgr2=0x563fc8daeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d42a00640 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8daeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d42a00640 1 -- 172.21.15.3:0/232883602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 msgr2=0x563fc8da24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d42a00640 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8da24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d42a00640 1 -- 172.21.15.3:0/232883602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc8dad1c0 con 0x563fc8da64e0 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d42a00640 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da64e0 0x563fc8dc14f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009a50 tx=0x7f1d3401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1272db181b50f183 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.675+0000 7f1d33400640 1 -- 172.21.15.3:0/232883602 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3401ebf0 con 0x563fc8da64e0 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d33400640 1 -- 172.21.15.3:0/232883602 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3401ed90 con 0x563fc8da64e0 2024-08-01T04:16:56.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d33400640 1 -- 172.21.15.3:0/232883602 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3401d910 con 0x563fc8da64e0 2024-08-01T04:16:56.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/232883602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da64e0 msgr2=0x563fc8dc14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da64e0 0x563fc8dc14f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009a50 tx=0x7f1d3401b000 comp rx=0 tx=0).stop 2024-08-01T04:16:56.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/232883602 shutdown_connections 2024-08-01T04:16:56.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8db1280 0x563fc8da24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da7a50 0x563fc8daeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/232883602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8da64e0 0x563fc8dc14f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/232883602 >> 172.21.15.3:0/232883602 conn(0x563fc8d6ea00 msgr2=0x563fc8db0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.676+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/232883602 shutdown_connections 2024-08-01T04:16:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.677+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/232883602 wait complete. 2024-08-01T04:16:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.677+0000 7f1d4c2c0740 1 -- start start 2024-08-01T04:16:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.677+0000 7f1d4c2c0740 1 Processor -- start 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8dcbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 0x563fc8dab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8dab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563fc8dc2eb0 con 0x563fc8da7a50 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563fc8dc27e0 con 0x563fc8da64e0 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563fc8dc3580 con 0x563fc8db1280 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d4c2c0740 10 client.0 _ll_register_callbacks cb 0x7f1d2c0d35a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:16:56.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d42000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 0x563fc8dab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d42000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 0x563fc8dab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39460/0 (socket says 172.21.15.3:39460) 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d42000640 1 -- 172.21.15.3:0/2792331538 learned_addr learned my addr 172.21.15.3:0/2792331538 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d42a00640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8dcbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.678+0000 7f1d43400640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8dab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42000640 1 -- 172.21.15.3:0/2792331538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 msgr2=0x563fc8dab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42000640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8dab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42000640 1 -- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 msgr2=0x563fc8dcbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42000640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8dcbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42000640 1 -- 172.21.15.3:0/2792331538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc8dc9620 con 0x563fc8da7a50 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42a00640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8dcbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d42000640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 0x563fc8dab210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c098840 tx=0x7f1d2c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f57364a7017b6e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d30c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:16:56.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d30c00640 1 -- 172.21.15.3:0/2792331538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c09c570 con 0x563fc8da7a50 2024-08-01T04:16:56.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d30c00640 1 -- 172.21.15.3:0/2792331538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1d2c09b2c0 con 0x563fc8da7a50 2024-08-01T04:16:56.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.679+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3401ed90 con 0x563fc8da7a50 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d30c00640 1 -- 172.21.15.3:0/2792331538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d2c09be70 con 0x563fc8da7a50 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 10 client.15216 fetch_fsmap learned FSMap version 26 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fc8e2c420 con 0x563fc8da7a50 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d30c00640 1 -- 172.21.15.3:0/2792331538 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3401ed90 con 0x563fc8da7a50 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d30c00640 1 -- 172.21.15.3:0/2792331538 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f1d2c097dc0 con 0x563fc8da7a50 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 20 client.15216 populate_metadata read hostname 'smithi003' 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8e31070 0x563fc8e2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 4 client.15216 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-01T04:16:56.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.680+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "status"}) -- 0x563fc8d143c0 con 0x563fc8e31070 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.681+0000 7f1d42a00640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8e31070 0x563fc8e2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.681+0000 7f1d42a00640 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8e31070 0x563fc8e2fd80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c19eceb63bd6a3d server_cookie=3ce967bb0b8d6ed5 in_seq=0 out_seq=0 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.681+0000 7f1d30c00640 10 client.15216 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d30c00640 1 -- 172.21.15.3:0/2792331538 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563fc8d143c0 con 0x563fc8e31070 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d30c00640 10 client.15216 handle_command_reply: tid=0 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 client.15216 shutdown 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 msgr2=0x563fc8dab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 0x563fc8dab210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c098840 tx=0x7f1d2c010f10 comp rx=0 tx=0).stop 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 shutdown_connections 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563fc8e31070 0x563fc8e2fd80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563fc8db1280 0x563fc8dab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563fc8da7a50 0x563fc8dab210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 --2- 172.21.15.3:0/2792331538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fc8da64e0 0x563fc8dcbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 >> 172.21.15.3:0/2792331538 conn(0x563fc8d6ea00 msgr2=0x563fc8dbc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d30c00640 0 client.15216 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 shutdown_connections 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 1 -- 172.21.15.3:0/2792331538 wait complete. 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.682+0000 7f1d4c2c0740 20 client.15216 trim_cache size 0 max 16384 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "2f6d75a6-4fbb-11ef-bcc9-c7b262605968", 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-08-01T04:16:56.686 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24548, 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 180.16221782299999, 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 26, 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 91, 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 195.20482113899999 2024-08-01T04:16:56.687 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d280a2e10 msgr2=0x7f1d280a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d280a2e10 0x7f1d280a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x563fc8da39d0 tx=0x7f1d34009260 comp rx=0 tx=0).stop 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 msgr2=0x7f1d4419ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 0x7f1d4419ea90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00e9f0 tx=0x7f1d2c00edf0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 shutdown_connections 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d4410dd90 0x7f1d441a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d441086b0 0x7f1d4419ea90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d280a2e10 0x7f1d280a52d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 --2- 172.21.15.3:0/3244151439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d44107d50 0x7f1d4419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 >> 172.21.15.3:0/3244151439 conn(0x7f1d440776b0 msgr2=0x7f1d44079060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:16:56.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 shutdown_connections 2024-08-01T04:16:56.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:16:56.683+0000 7f1d43e00640 1 -- 172.21.15.3:0/3244151439 wait complete. 2024-08-01T04:16:57.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:56 smithi163 ceph-mon[37383]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:16:57.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:56 smithi003 ceph-mon[32262]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:16:57.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:56 smithi160 ceph-mon[37224]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:16:57.560 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-01T04:16:59.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:16:58 smithi163 ceph-mon[37383]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-01T04:16:59.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:16:58 smithi003 ceph-mon[32262]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-01T04:16:59.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:16:58 smithi160 ceph-mon[37224]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-01T04:17:00.549 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:17:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:17:00.046+0000 7fefda000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:17:00.549 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:17:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:17:00.046+0000 7fefda000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:17:00.549 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:17:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:17:00.046+0000 7fefda000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:17:00.549 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:17:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:17:00.046+0000 7fefda000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:17:00.549 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:17:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:17:00.046+0000 7fefda000640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-01T04:17:01.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:00 smithi163 ceph-mon[37383]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:17:01.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:00 smithi003 ceph-mon[32262]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:17:01.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:00 smithi160 ceph-mon[37224]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:17:02.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:17:03.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:03 smithi003 ceph-mon[32262]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-08-01T04:17:03.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:03 smithi003 ceph-mon[32262]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-08-01T04:17:03.177 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:03.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:03 smithi163 ceph-mon[37383]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-08-01T04:17:03.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:03 smithi163 ceph-mon[37383]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-08-01T04:17:03.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:03 smithi160 ceph-mon[37224]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-08-01T04:17:03.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:03 smithi160 ceph-mon[37224]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-08-01T04:17:03.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 -- 172.21.15.3:0/2878365566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074b60 msgr2=0x7fa6a010da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 --2- 172.21.15.3:0/2878365566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074b60 0x7fa6a010da70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009900 tx=0x7fa68c02a480 comp rx=0 tx=0).stop 2024-08-01T04:17:03.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 -- 172.21.15.3:0/2878365566 shutdown_connections 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 --2- 172.21.15.3:0/2878365566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6a010dfb0 0x7fa6a01103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 --2- 172.21.15.3:0/2878365566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074b60 0x7fa6a010da70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 --2- 172.21.15.3:0/2878365566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a0074240 0x7fa6a0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 -- 172.21.15.3:0/2878365566 >> 172.21.15.3:0/2878365566 conn(0x7fa6a00fd470 msgr2=0x7fa6a00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 -- 172.21.15.3:0/2878365566 shutdown_connections 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.877+0000 7fa69f200640 1 -- 172.21.15.3:0/2878365566 wait complete. 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.878+0000 7fa69f200640 1 Processor -- start 2024-08-01T04:17:03.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 -- start start 2024-08-01T04:17:03.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 0x7fa6a019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6a0074b60 0x7fa6a019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a010dfb0 0x7fa6a019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0113d30 con 0x7fa6a010dfb0 2024-08-01T04:17:03.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0113bb0 con 0x7fa6a0074240 2024-08-01T04:17:03.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0113eb0 con 0x7fa6a0074b60 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 0x7fa6a019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 0x7fa6a019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51436/0 (socket says 172.21.15.3:51436) 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69de00640 1 -- 172.21.15.3:0/3871556596 learned_addr learned my addr 172.21.15.3:0/3871556596 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69d400640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6a0074b60 0x7fa6a019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.879+0000 7fa69e800640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a010dfb0 0x7fa6a019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69de00640 1 -- 172.21.15.3:0/3871556596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6a0074b60 msgr2=0x7fa6a019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69de00640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6a0074b60 0x7fa6a019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69de00640 1 -- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a010dfb0 msgr2=0x7fa6a019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69de00640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a010dfb0 0x7fa6a019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69de00640 1 -- 172.21.15.3:0/3871556596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a019f1e0 con 0x7fa6a0074240 2024-08-01T04:17:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.880+0000 7fa69de00640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 0x7fa6a019a230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa69000e9f0 tx=0x7fa69000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.881+0000 7fa696a00640 1 -- 172.21.15.3:0/3871556596 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000cd80 con 0x7fa6a0074240 2024-08-01T04:17:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.881+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a019f4d0 con 0x7fa6a0074240 2024-08-01T04:17:03.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.881+0000 7fa696a00640 1 -- 172.21.15.3:0/3871556596 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690010910 con 0x7fa6a0074240 2024-08-01T04:17:03.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.881+0000 7fa696a00640 1 -- 172.21.15.3:0/3871556596 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900188f0 con 0x7fa6a0074240 2024-08-01T04:17:03.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.881+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a01a6db0 con 0x7fa6a0074240 2024-08-01T04:17:03.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 Processor -- start 2024-08-01T04:17:03.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 -- start start 2024-08-01T04:17:03.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f170160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f1706a0 0x55c62f17aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c62f0d5bf0 con 0x55c62f17af60 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c62f06dbd0 con 0x55c62f16fd80 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.885+0000 7fa6a7409740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c62f0d1bb0 con 0x55c62f1706a0 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f170160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f170160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51438/0 (socket says 172.21.15.3:51438) 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 -- 172.21.15.3:0/2366240035 learned_addr learned my addr 172.21.15.3:0/2366240035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:03.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69d400640 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 -- 172.21.15.3:0/2366240035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f1706a0 msgr2=0x55c62f17aa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f1706a0 0x55c62f17aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 -- 172.21.15.3:0/2366240035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 msgr2=0x55c62f17d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa69e800640 1 -- 172.21.15.3:0/2366240035 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62f0748a0 con 0x55c62f16fd80 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.886+0000 7fa696a00640 1 -- 172.21.15.3:0/3871556596 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7fa690010ac0 con 0x7fa6a0074240 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.887+0000 7fa69e800640 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f170160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa698009e30 tx=0x7fa698010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c8afdcf401a653 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.887+0000 7fa68b000640 1 -- 172.21.15.3:0/2366240035 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698016070 con 0x55c62f16fd80 2024-08-01T04:17:03.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.887+0000 7fa68b000640 1 -- 172.21.15.3:0/2366240035 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa698004d20 con 0x55c62f16fd80 2024-08-01T04:17:03.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.887+0000 7fa68b000640 1 -- 172.21.15.3:0/2366240035 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698012850 con 0x55c62f16fd80 2024-08-01T04:17:03.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 -- 172.21.15.3:0/2366240035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 msgr2=0x55c62f170160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f170160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa698009e30 tx=0x7fa698010740 comp rx=0 tx=0).stop 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 -- 172.21.15.3:0/2366240035 shutdown_connections 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f1706a0 0x55c62f17aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 --2- 172.21.15.3:0/2366240035 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f170160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 -- 172.21.15.3:0/2366240035 >> 172.21.15.3:0/2366240035 conn(0x55c62f16c310 msgr2=0x55c62f16c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 -- 172.21.15.3:0/2366240035 shutdown_connections 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.888+0000 7fa6a7409740 1 -- 172.21.15.3:0/2366240035 wait complete. 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.889+0000 7fa6a7409740 1 -- start start 2024-08-01T04:17:03.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.889+0000 7fa696a00640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6840a2e40 0x7fa6840a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.889+0000 7fa696a00640 1 -- 172.21.15.3:0/3871556596 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa6900cfc30 con 0x7fa6a0074240 2024-08-01T04:17:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.889+0000 7fa6a7409740 1 Processor -- start 2024-08-01T04:17:03.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.889+0000 7fa69e800640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6840a2e40 0x7fa6840a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f1797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 0x55c62f179ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c62f17db30 con 0x55c62f17af60 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c62f090d40 con 0x55c62f1706a0 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c62f17dcb0 con 0x55c62f16fd80 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa6a7409740 10 client.0 _ll_register_callbacks cb 0x55c62f059b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa69d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 0x55c62f179ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa69d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 0x55c62f179ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51444/0 (socket says 172.21.15.3:51444) 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.890+0000 7fa69d400640 1 -- 172.21.15.3:0/984593752 learned_addr learned my addr 172.21.15.3:0/984593752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f1797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69e800640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 -- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 msgr2=0x55c62f179ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 0x55c62f179ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 -- 172.21.15.3:0/984593752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 msgr2=0x55c62f17a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 -- 172.21.15.3:0/984593752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62ee473c0 con 0x55c62f16fd80 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69d400640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 0x55c62f179ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69e800640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa69de00640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f1797a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6900cd100 tx=0x7fa69009bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9abbfa26eb5aae89 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.891+0000 7fa67f400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa69e800640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6840a2e40 0x7fa6840a5300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa698009bc0 tx=0x7fa698005fd0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa67f400640 1 -- 172.21.15.3:0/984593752 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900a0280 con 0x55c62f16fd80 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa67f400640 1 -- 172.21.15.3:0/984593752 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa690010430 con 0x55c62f16fd80 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa698004d20 con 0x55c62f16fd80 2024-08-01T04:17:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa67f400640 1 -- 172.21.15.3:0/984593752 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6900a4b40 con 0x55c62f16fd80 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa6a7409740 10 client.25351 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.892+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c62f1920b0 con 0x55c62f16fd80 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa67f400640 1 -- 172.21.15.3:0/984593752 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa698004d20 con 0x55c62f16fd80 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa67f400640 1 -- 172.21.15.3:0/984593752 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa69009ae20 con 0x55c62f16fd80 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa6a7409740 10 client.25351 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa6a7409740 10 client.25351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa6a7409740 20 client.25351 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa6a7409740 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f193570 0x55c62f195980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa6a7409740 4 client.25351 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.893+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c62f0e3160 con 0x55c62f193570 2024-08-01T04:17:03.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.894+0000 7fa69d400640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f193570 0x55c62f195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.894+0000 7fa69d400640 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f193570 0x55c62f195980 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a27b6cd222d77d server_cookie=1070c4d5f552ba60 in_seq=0 out_seq=0 2024-08-01T04:17:03.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.894+0000 7fa67f400640 10 client.25351 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa67f400640 1 -- 172.21.15.3:0/984593752 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c62f0e3160 con 0x55c62f193570 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa67f400640 10 client.25351 handle_command_reply: tid=0 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa6a7409740 1 client.25351 shutdown 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 msgr2=0x55c62f1797a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa6a7409740 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f1797a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6900cd100 tx=0x7fa69009bb00 comp rx=0 tx=0).stop 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 shutdown_connections 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa6a7409740 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f193570 0x55c62f195980 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.895+0000 7fa6a7409740 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f17af60 0x55c62f17a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa67f400640 0 client.25351 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa6a7409740 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f1706a0 0x55c62f179ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa6a7409740 1 --2- 172.21.15.3:0/984593752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f1797a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 >> 172.21.15.3:0/984593752 conn(0x55c62f16c310 msgr2=0x55c62f171ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 shutdown_connections 2024-08-01T04:17:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa6a7409740 1 -- 172.21.15.3:0/984593752 wait complete. 2024-08-01T04:17:03.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.896+0000 7fa6a7409740 20 client.25351 trim_cache size 0 max 16384 2024-08-01T04:17:03.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 Processor -- start 2024-08-01T04:17:03.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 -- start start 2024-08-01T04:17:03.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f171ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f19b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f17af60 0x55c62f19b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01add70 con 0x55c62f1706a0 2024-08-01T04:17:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01ab640 con 0x55c62f17af60 2024-08-01T04:17:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.915+0000 7fa6a7409740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01a94b0 con 0x55c62f16fd80 2024-08-01T04:17:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f171ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f171ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51886/0 (socket says 172.21.15.3:51886) 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 -- 172.21.15.3:0/1406866839 learned_addr learned my addr 172.21.15.3:0/1406866839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69e800640 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f17af60 0x55c62f19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 -- 172.21.15.3:0/1406866839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f17af60 msgr2=0x55c62f19b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f17af60 0x55c62f19b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 -- 172.21.15.3:0/1406866839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 msgr2=0x55c62f19b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f19b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.916+0000 7fa69de00640 1 -- 172.21.15.3:0/1406866839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62f17d680 con 0x55c62f16fd80 2024-08-01T04:17:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.917+0000 7fa69de00640 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f171ee0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa69009a180 tx=0x7fa69009c050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac40a308de53468 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.917+0000 7fa68a600640 1 -- 172.21.15.3:0/1406866839 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69009ad90 con 0x55c62f16fd80 2024-08-01T04:17:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.917+0000 7fa68a600640 1 -- 172.21.15.3:0/1406866839 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69009af30 con 0x55c62f16fd80 2024-08-01T04:17:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.917+0000 7fa68a600640 1 -- 172.21.15.3:0/1406866839 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900cd690 con 0x55c62f16fd80 2024-08-01T04:17:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 -- 172.21.15.3:0/1406866839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 msgr2=0x55c62f171ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f171ee0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa69009a180 tx=0x7fa69009c050 comp rx=0 tx=0).stop 2024-08-01T04:17:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 -- 172.21.15.3:0/1406866839 shutdown_connections 2024-08-01T04:17:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f17af60 0x55c62f19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f19b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 --2- 172.21.15.3:0/1406866839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f16fd80 0x55c62f171ee0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 -- 172.21.15.3:0/1406866839 >> 172.21.15.3:0/1406866839 conn(0x55c62f16c310 msgr2=0x55c62f1717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 -- 172.21.15.3:0/1406866839 shutdown_connections 2024-08-01T04:17:03.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.918+0000 7fa6a7409740 1 -- 172.21.15.3:0/1406866839 wait complete. 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.919+0000 7fa6a7409740 1 -- start start 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.919+0000 7fa6a7409740 1 Processor -- start 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f187810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f187d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f17af60 0x55c62f1882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01a9b00 con 0x55c62f1706a0 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01aaf70 con 0x55c62f16fd80 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01aa1d0 con 0x55c62f17af60 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa6a7409740 10 client.0 _ll_register_callbacks cb 0x7fa69000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:03.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f187d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f187d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37028/0 (socket says 172.21.15.3:37028) 2024-08-01T04:17:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 -- 172.21.15.3:0/3461850418 learned_addr learned my addr 172.21.15.3:0/3461850418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69de00640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f187810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69e800640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f17af60 0x55c62f1882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 -- 172.21.15.3:0/3461850418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f17af60 msgr2=0x55c62f1882b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f17af60 0x55c62f1882b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 -- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 msgr2=0x55c62f187810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.920+0000 7fa69d400640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f187810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa69d400640 1 -- 172.21.15.3:0/3461850418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62f1fd4c0 con 0x55c62f1706a0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa69de00640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f187810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa69d400640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f187d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009850 tx=0x7fa68c004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13431dda69adc612 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa67f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa67f400640 1 -- 172.21.15.3:0/3461850418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c035040 con 0x55c62f1706a0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69009af30 con 0x55c62f1706a0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.921+0000 7fa67f400640 1 -- 172.21.15.3:0/3461850418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa68c005510 con 0x55c62f1706a0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa67f400640 1 -- 172.21.15.3:0/3461850418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa68c0046c0 con 0x55c62f1706a0 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 10 client.15240 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:03.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c62f1fe830 con 0x55c62f1706a0 2024-08-01T04:17:03.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa67f400640 1 -- 172.21.15.3:0/3461850418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69009af30 con 0x55c62f1706a0 2024-08-01T04:17:03.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa67f400640 1 -- 172.21.15.3:0/3461850418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa68c031b70 con 0x55c62f1706a0 2024-08-01T04:17:03.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:03.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 10 client.15240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:03.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 20 client.15240 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f1fef60 0x55c62f201380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 4 client.15240 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:17:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.922+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c62f0dd670 con 0x55c62f1fef60 2024-08-01T04:17:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.923+0000 7fa69de00640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f1fef60 0x55c62f201380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa69de00640 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f1fef60 0x55c62f201380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a101221f1ecc6c4d server_cookie=25ca15bb5589a31d in_seq=0 out_seq=0 2024-08-01T04:17:03.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa67f400640 10 client.15240 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa67f400640 1 -- 172.21.15.3:0/3461850418 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c62f0dd670 con 0x55c62f1fef60 2024-08-01T04:17:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa67f400640 10 client.15240 handle_command_reply: tid=0 2024-08-01T04:17:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 client.15240 shutdown 2024-08-01T04:17:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 msgr2=0x55c62f187d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f187d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009850 tx=0x7fa68c004560 comp rx=0 tx=0).stop 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 shutdown_connections 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c62f1fef60 0x55c62f201380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c62f17af60 0x55c62f1882b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c62f1706a0 0x55c62f187d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 --2- 172.21.15.3:0/3461850418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c62f16fd80 0x55c62f187810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 >> 172.21.15.3:0/3461850418 conn(0x55c62f16c310 msgr2=0x55c62f179fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.924+0000 7fa67f400640 0 client.15240 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.925+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 shutdown_connections 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.925+0000 7fa6a7409740 1 -- 172.21.15.3:0/3461850418 wait complete. 2024-08-01T04:17:03.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.925+0000 7fa6a7409740 20 client.15240 trim_cache size 0 max 16384 2024-08-01T04:17:03.928 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:03.928 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:17:03.928 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:17:03.928 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:03.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6840a2e40 msgr2=0x7fa6840a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6840a2e40 0x7fa6840a5300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa698009bc0 tx=0x7fa698005fd0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 msgr2=0x7fa6a019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 0x7fa6a019a230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa69000e9f0 tx=0x7fa69000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 shutdown_connections 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a010dfb0 0x7fa6a019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6840a2e40 0x7fa6840a5300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6a0074b60 0x7fa6a019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 --2- 172.21.15.3:0/3871556596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6a0074240 0x7fa6a019a230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 >> 172.21.15.3:0/3871556596 conn(0x7fa6a00fd470 msgr2=0x7fa6a00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 shutdown_connections 2024-08-01T04:17:03.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:03.926+0000 7fa69f200640 1 -- 172.21.15.3:0/3871556596 wait complete. 2024-08-01T04:17:04.964 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:17:04.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:17:05.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:05 smithi003 ceph-mon[32262]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:05.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:05 smithi163 ceph-mon[37383]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:05.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:05 smithi160 ceph-mon[37224]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:05.483 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:06.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 -- 172.21.15.3:0/3271971645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c104460 msgr2=0x7f750c110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 --2- 172.21.15.3:0/3271971645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c104460 0x7f750c110990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7504009cd0 tx=0x7f750402a8e0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 -- 172.21.15.3:0/3271971645 shutdown_connections 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 --2- 172.21.15.3:0/3271971645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c104460 0x7f750c110990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 --2- 172.21.15.3:0/3271971645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f750c103ac0 0x7f750c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 --2- 172.21.15.3:0/3271971645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c109ac0 0x7f750c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 -- 172.21.15.3:0/3271971645 >> 172.21.15.3:0/3271971645 conn(0x7f750c0ff7c0 msgr2=0x7f750c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 -- 172.21.15.3:0/3271971645 shutdown_connections 2024-08-01T04:17:06.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.208+0000 7f7514400640 1 -- 172.21.15.3:0/3271971645 wait complete. 2024-08-01T04:17:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.209+0000 7f7514400640 1 Processor -- start 2024-08-01T04:17:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.209+0000 7f7514400640 1 -- start start 2024-08-01T04:17:06.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7514400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 0x7f750c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7514400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c104460 0x7f750c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7514400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f750c109ac0 0x7f750c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7514400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f750c113f10 con 0x7f750c103ac0 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7514400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f750c113d90 con 0x7f750c104460 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7514400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f750c114090 con 0x7f750c109ac0 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7511a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 0x7f750c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7511a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 0x7f750c19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37044/0 (socket says 172.21.15.3:37044) 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7511a00640 1 -- 172.21.15.3:0/4173747707 learned_addr learned my addr 172.21.15.3:0/4173747707 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.210+0000 7f7512400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f750c109ac0 0x7f750c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:06.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511000640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c104460 0x7f750c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511a00640 1 -- 172.21.15.3:0/4173747707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f750c109ac0 msgr2=0x7f750c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511a00640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f750c109ac0 0x7f750c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511a00640 1 -- 172.21.15.3:0/4173747707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c104460 msgr2=0x7f750c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511a00640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c104460 0x7f750c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511a00640 1 -- 172.21.15.3:0/4173747707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c19f2f0 con 0x7f750c103ac0 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.211+0000 7f7511a00640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 0x7f750c19a340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f750000ea60 tx=0x7f750000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:06.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.212+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750000cde0 con 0x7f750c103ac0 2024-08-01T04:17:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.212+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75000035f0 con 0x7f750c103ac0 2024-08-01T04:17:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.212+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c19f5e0 con 0x7f750c103ac0 2024-08-01T04:17:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.212+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c1a6ec0 con 0x7f750c103ac0 2024-08-01T04:17:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.212+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500010670 con 0x7f750c103ac0 2024-08-01T04:17:06.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.214+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c103f20 con 0x7f750c103ac0 2024-08-01T04:17:06.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.215+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f7500004020 con 0x7f750c103ac0 2024-08-01T04:17:06.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.217+0000 7f750a000640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f74f40a2f20 0x7f74f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:06.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.218+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f75000cf910 con 0x7f750c103ac0 2024-08-01T04:17:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.218+0000 7f7511000640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f74f40a2f20 0x7f74f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:06.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.219+0000 7f7511000640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f74f40a2f20 0x7f74f40a53e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f750c101cf0 tx=0x7f74f8005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:06.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.221+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f750009c2b0 con 0x7f750c103ac0 2024-08-01T04:17:06.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.459+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f750c1a7150 con 0x7f750c103ac0 2024-08-01T04:17:06.462 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:17:06.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.460+0000 7f750a000640 1 -- 172.21.15.3:0/4173747707 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7f750009bbb0 con 0x7f750c103ac0 2024-08-01T04:17:06.462 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:17:06.463 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:17:06.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.463+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f74f40a2f20 msgr2=0x7f74f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:06.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.463+0000 7f7514400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f74f40a2f20 0x7f74f40a53e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f750c101cf0 tx=0x7f74f8005f90 comp rx=0 tx=0).stop 2024-08-01T04:17:06.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.463+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 msgr2=0x7f750c19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:06.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.463+0000 7f7514400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 0x7f750c19a340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f750000ea60 tx=0x7f750000ef20 comp rx=0 tx=0).stop 2024-08-01T04:17:06.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 shutdown_connections 2024-08-01T04:17:06.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f74f40a2f20 0x7f74f40a53e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f750c109ac0 0x7f750c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f750c104460 0x7f750c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 --2- 172.21.15.3:0/4173747707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f750c103ac0 0x7f750c19a340 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:06.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 >> 172.21.15.3:0/4173747707 conn(0x7f750c0ff7c0 msgr2=0x7f750c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:06.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 shutdown_connections 2024-08-01T04:17:06.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:06.464+0000 7f7514400640 1 -- 172.21.15.3:0/4173747707 wait complete. 2024-08-01T04:17:07.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:07 smithi163 ceph-mon[37383]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:07.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:07 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/4173747707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:07.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:07 smithi003 ceph-mon[32262]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:07.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:07 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/4173747707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:07.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:07 smithi160 ceph-mon[37224]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:07.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:07 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/4173747707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:07.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:17:07.911 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 -- 172.21.15.3:0/1304138297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f400810df90 msgr2=0x7f4008110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 --2- 172.21.15.3:0/1304138297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f400810df90 0x7f4008110380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4000009ca0 tx=0x7f400002a950 comp rx=0 tx=0).stop 2024-08-01T04:17:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 -- 172.21.15.3:0/1304138297 shutdown_connections 2024-08-01T04:17:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 --2- 172.21.15.3:0/1304138297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f400810df90 0x7f4008110380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 --2- 172.21.15.3:0/1304138297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4008074c10 0x7f400810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 --2- 172.21.15.3:0/1304138297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 0x7f40080746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 -- 172.21.15.3:0/1304138297 >> 172.21.15.3:0/1304138297 conn(0x7f40080fd470 msgr2=0x7f40080ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:08.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.606+0000 7f4007e00640 1 -- 172.21.15.3:0/1304138297 shutdown_connections 2024-08-01T04:17:08.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.607+0000 7f4007e00640 1 -- 172.21.15.3:0/1304138297 wait complete. 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.608+0000 7f4007e00640 1 Processor -- start 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.608+0000 7f4007e00640 1 -- start start 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.608+0000 7f4007e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 0x7f400819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.608+0000 7f4007e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 0x7f400819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.608+0000 7f4007e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400810df90 0x7f400819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.608+0000 7f4007e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4008113d20 con 0x7f4008074c10 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f4007e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4008113ba0 con 0x7f40080742f0 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f4007e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4008113ea0 con 0x7f400810df90 2024-08-01T04:17:08.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 0x7f400819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f4006a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 0x7f400819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 0x7f400819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37052/0 (socket says 172.21.15.3:37052) 2024-08-01T04:17:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 -- 172.21.15.3:0/3822324174 learned_addr learned my addr 172.21.15.3:0/3822324174 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:08.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f4007400640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400810df90 0x7f400819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 -- 172.21.15.3:0/3822324174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400810df90 msgr2=0x7f400819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400810df90 0x7f400819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 -- 172.21.15.3:0/3822324174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 msgr2=0x7f400819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 0x7f400819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.609+0000 7f3fffe00640 1 -- 172.21.15.3:0/3822324174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400819f180 con 0x7f4008074c10 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f4007400640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400810df90 0x7f400819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f4006a00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 0x7f400819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 0x7f400819a710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e9f0 tx=0x7f3ff000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f3fff400640 1 -- 172.21.15.3:0/3822324174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000cd80 con 0x7f4008074c10 2024-08-01T04:17:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f3fff400640 1 -- 172.21.15.3:0/3822324174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff0005500 con 0x7f4008074c10 2024-08-01T04:17:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f3fff400640 1 -- 172.21.15.3:0/3822324174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010690 con 0x7f4008074c10 2024-08-01T04:17:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.610+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400819f470 con 0x7f4008074c10 2024-08-01T04:17:08.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.611+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40081a6d50 con 0x7f4008074c10 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 Processor -- start 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 -- start start 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556e5610bbf0 con 0x556e561a66e0 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556e560a3b20 con 0x556e561a7000 2024-08-01T04:17:08.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.614+0000 7f40102c8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556e56107bb0 con 0x556e561b18c0 2024-08-01T04:17:08.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51506/0 (socket says 172.21.15.3:51506) 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 -- 172.21.15.3:0/2637854994 learned_addr learned my addr 172.21.15.3:0/2637854994 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4007400640 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 -- 172.21.15.3:0/2637854994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 msgr2=0x556e561b3cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561b3cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 -- 172.21.15.3:0/2637854994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 msgr2=0x556e561a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 -- 172.21.15.3:0/2637854994 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e560aa7f0 con 0x556e561a7000 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f3fff400640 1 -- 172.21.15.3:0/3822324174 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f3ff00046c0 con 0x7f4008074c10 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.615+0000 7f4006a00640 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561b1380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8011540 tx=0x7f3ff8011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caf39df3544daa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.616+0000 7f3ff7e00640 1 -- 172.21.15.3:0/2637854994 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800f640 con 0x556e561a7000 2024-08-01T04:17:08.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.616+0000 7f3ff7e00640 1 -- 172.21.15.3:0/2637854994 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff800fcc0 con 0x556e561a7000 2024-08-01T04:17:08.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.616+0000 7f3ff7e00640 1 -- 172.21.15.3:0/2637854994 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8019e60 con 0x556e561a7000 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.616+0000 7f40102c8740 1 -- 172.21.15.3:0/2637854994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 msgr2=0x556e561b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.616+0000 7f40102c8740 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561b1380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8011540 tx=0x7f3ff8011a00 comp rx=0 tx=0).stop 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 -- 172.21.15.3:0/2637854994 shutdown_connections 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561b1380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 --2- 172.21.15.3:0/2637854994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 -- 172.21.15.3:0/2637854994 >> 172.21.15.3:0/2637854994 conn(0x556e561a2c70 msgr2=0x556e561a3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 -- 172.21.15.3:0/2637854994 shutdown_connections 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f40102c8740 1 -- 172.21.15.3:0/2637854994 wait complete. 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.617+0000 7f3fff400640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3fec0a2e30 0x7f3fec0a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.618+0000 7f40102c8740 1 -- start start 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.618+0000 7f3fff400640 1 -- 172.21.15.3:0/3822324174 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3ff00cfac0 con 0x7f4008074c10 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.618+0000 7f4007400640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3fec0a2e30 0x7f3fec0a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.618+0000 7f40102c8740 1 Processor -- start 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 0x556e561ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a7000 0x556e561aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556e561b4490 con 0x556e561a7000 2024-08-01T04:17:08.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556e560c6cf0 con 0x556e561a66e0 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556e561b4610 con 0x556e561b18c0 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f4007400640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3fec0a2e30 0x7f3fec0a52f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4000004af0 tx=0x7f4000005d80 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f40102c8740 10 client.0 _ll_register_callbacks cb 0x556e560b2e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a7000 0x556e561aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f4006a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 0x556e561ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f4006a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 0x556e561ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51510/0 (socket says 172.21.15.3:51510) 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f4006a00640 1 -- 172.21.15.3:0/191328390 learned_addr learned my addr 172.21.15.3:0/191328390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a7000 0x556e561aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37078/0 (socket says 172.21.15.3:37078) 2024-08-01T04:17:08.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.619+0000 7f4007400640 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f4006a00640 1 -- 172.21.15.3:0/191328390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 msgr2=0x556e561af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f4006a00640 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f4006a00640 1 -- 172.21.15.3:0/191328390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a7000 msgr2=0x556e561aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f4006a00640 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a7000 0x556e561aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f4006a00640 1 -- 172.21.15.3:0/191328390 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e55e7d3c0 con 0x556e561a66e0 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f4006a00640 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 0x556e561ae830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ff801b040 tx=0x7f3ff8019dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6baf8f63f758781 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f3ff4c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f3ff4c00640 1 -- 172.21.15.3:0/191328390 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8027060 con 0x556e561a66e0 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff800fcc0 con 0x556e561a66e0 2024-08-01T04:17:08.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.620+0000 7f3ff4c00640 1 -- 172.21.15.3:0/191328390 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3ff800f850 con 0x556e561a66e0 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.621+0000 7f3ff4c00640 1 -- 172.21.15.3:0/191328390 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ff800fa50 con 0x556e561a66e0 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.621+0000 7f40102c8740 10 client.25391 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.621+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e561c89d0 con 0x556e561a66e0 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.621+0000 7f3ff4c00640 1 -- 172.21.15.3:0/191328390 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff800fcc0 con 0x556e561a66e0 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.621+0000 7f3ff4c00640 1 -- 172.21.15.3:0/191328390 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff8004a90 con 0x556e561a66e0 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f40102c8740 10 client.25391 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f40102c8740 10 client.25391 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f40102c8740 20 client.25391 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f40102c8740 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e561cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f40102c8740 4 client.25391 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e56119160 con 0x556e561c9d80 2024-08-01T04:17:08.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f3fffe00640 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e561cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.622+0000 7f3fffe00640 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e561cc170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68187a301d9b94e5 server_cookie=7521e998b658919f in_seq=0 out_seq=0 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.623+0000 7f3ff4c00640 10 client.25391 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.623+0000 7f3ff4c00640 1 -- 172.21.15.3:0/191328390 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e56119160 con 0x556e561c9d80 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.623+0000 7f3ff4c00640 10 client.25391 handle_command_reply: tid=0 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.623+0000 7f40102c8740 1 client.25391 shutdown 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 msgr2=0x556e561ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 0x556e561ae830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ff801b040 tx=0x7f3ff8019dd0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 shutdown_connections 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e561cc170 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f3ff4c00640 0 client.25391 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:08.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a7000 0x556e561aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 --2- 172.21.15.3:0/191328390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a66e0 0x556e561ae830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 >> 172.21.15.3:0/191328390 conn(0x556e561a2c70 msgr2=0x556e561a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:08.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 shutdown_connections 2024-08-01T04:17:08.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 1 -- 172.21.15.3:0/191328390 wait complete. 2024-08-01T04:17:08.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.624+0000 7f40102c8740 20 client.25391 trim_cache size 0 max 16384 2024-08-01T04:17:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.652+0000 7f40102c8740 1 Processor -- start 2024-08-01T04:17:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.652+0000 7f40102c8740 1 -- start start 2024-08-01T04:17:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f40102c8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561a66e0 0x556e561b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f40102c8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f40102c8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561b18c0 0x556e561d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f40102c8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40081adc60 con 0x556e561b18c0 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f40102c8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f40081ab530 con 0x556e561a7000 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f40102c8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f40081a9380 con 0x556e561a66e0 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f4006a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561a66e0 0x556e561b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f4006a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561a66e0 0x556e561b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51952/0 (socket says 172.21.15.3:51952) 2024-08-01T04:17:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f3fffe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51520/0 (socket says 172.21.15.3:51520) 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f4006a00640 1 -- 172.21.15.3:0/4183594042 learned_addr learned my addr 172.21.15.3:0/4183594042 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.653+0000 7f4007400640 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561b18c0 0x556e561d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3fffe00640 1 -- 172.21.15.3:0/4183594042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561a66e0 msgr2=0x556e561b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3fffe00640 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561a66e0 0x556e561b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3fffe00640 1 -- 172.21.15.3:0/4183594042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561b18c0 msgr2=0x556e561d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3fffe00640 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561b18c0 0x556e561d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3fffe00640 1 -- 172.21.15.3:0/4183594042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e561a6460 con 0x556e561a7000 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3fffe00640 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561d1860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00ccf90 tx=0x7f3ff00980d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a528663c6a00f7f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.654+0000 7f3ff7400640 1 -- 172.21.15.3:0/4183594042 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00b8030 con 0x556e561a7000 2024-08-01T04:17:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f3ff7400640 1 -- 172.21.15.3:0/4183594042 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff00a0110 con 0x556e561a7000 2024-08-01T04:17:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f3ff7400640 1 -- 172.21.15.3:0/4183594042 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00a47a0 con 0x556e561a7000 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 -- 172.21.15.3:0/4183594042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 msgr2=0x556e561d1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561d1860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00ccf90 tx=0x7f3ff00980d0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 -- 172.21.15.3:0/4183594042 shutdown_connections 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561b18c0 0x556e561d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561d1860 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 --2- 172.21.15.3:0/4183594042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561a66e0 0x556e561b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.655+0000 7f40102c8740 1 -- 172.21.15.3:0/4183594042 >> 172.21.15.3:0/4183594042 conn(0x556e561a2c70 msgr2=0x556e561b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.656+0000 7f40102c8740 1 -- 172.21.15.3:0/4183594042 shutdown_connections 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.656+0000 7f40102c8740 1 -- 172.21.15.3:0/4183594042 wait complete. 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.656+0000 7f40102c8740 1 -- start start 2024-08-01T04:17:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.656+0000 7f40102c8740 1 Processor -- start 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561c2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561aae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40081a99f0 con 0x556e561a66e0 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f40081aae60 con 0x556e561a7000 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f40081aa0c0 con 0x556e561b18c0 2024-08-01T04:17:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f40102c8740 10 client.0 _ll_register_callbacks cb 0x7f3ff80197c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f4007400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f4006a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f4007400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561aae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51962/0 (socket says 172.21.15.3:51962) 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f4006a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561c22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37102/0 (socket says 172.21.15.3:37102) 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f4007400640 1 -- 172.21.15.3:0/3756795263 learned_addr learned my addr 172.21.15.3:0/3756795263 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.657+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3fffe00640 1 -- 172.21.15.3:0/3756795263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 msgr2=0x556e561aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3fffe00640 1 -- 172.21.15.3:0/3756795263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 msgr2=0x556e561c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3fffe00640 1 -- 172.21.15.3:0/3756795263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e561ab4f0 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f4006a00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3fffe00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561c2810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ff009c610 tx=0x7f3ff00a4a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84c108632756b35 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3ff4c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3ff4c00640 1 -- 172.21.15.3:0/3756795263 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00b8030 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f3ff4c00640 1 -- 172.21.15.3:0/3756795263 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3ff00a4540 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.658+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff00a0110 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f3ff4c00640 1 -- 172.21.15.3:0/3756795263 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ff00b0020 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 10 client.25397 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e56234190 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f3ff4c00640 1 -- 172.21.15.3:0/3756795263 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff00a0110 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f3ff4c00640 1 -- 172.21.15.3:0/3756795263 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3ff00b2040 con 0x556e561a7000 2024-08-01T04:17:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 10 client.25397 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 10 client.25397 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 20 client.25397 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e56236950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 4 client.25397 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.659+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e56113670 con 0x556e561c9d80 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.660+0000 7f4006a00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e56236950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.660+0000 7f4006a00640 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e56236950 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69fb466237a9fee server_cookie=395d82ab6ee64637 in_seq=0 out_seq=0 2024-08-01T04:17:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.660+0000 7f3ff4c00640 10 client.25397 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.660+0000 7f3ff4c00640 1 -- 172.21.15.3:0/3756795263 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e56113670 con 0x556e561c9d80 2024-08-01T04:17:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.660+0000 7f3ff4c00640 10 client.25397 handle_command_reply: tid=0 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.660+0000 7f40102c8740 1 client.25397 shutdown 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 msgr2=0x556e561c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561c2810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3ff009c610 tx=0x7f3ff00a4a50 comp rx=0 tx=0).stop 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 shutdown_connections 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x556e561c9d80 0x556e56236950 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556e561b18c0 0x556e561aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556e561a7000 0x556e561c2810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 --2- 172.21.15.3:0/3756795263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556e561a66e0 0x556e561c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f3ff4c00640 0 client.25397 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 >> 172.21.15.3:0/3756795263 conn(0x556e561a2c70 msgr2=0x556e561a1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 shutdown_connections 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 1 -- 172.21.15.3:0/3756795263 wait complete. 2024-08-01T04:17:08.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.661+0000 7f40102c8740 20 client.25397 trim_cache size 0 max 16384 2024-08-01T04:17:08.666 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:17:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3fec0a2e30 msgr2=0x7f3fec0a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3fec0a2e30 0x7f3fec0a52f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4000004af0 tx=0x7f4000005d80 comp rx=0 tx=0).stop 2024-08-01T04:17:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 msgr2=0x7f400819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:08.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 0x7f400819a710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e9f0 tx=0x7f3ff000edf0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 shutdown_connections 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3fec0a2e30 0x7f3fec0a52f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400810df90 0x7f400819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4008074c10 0x7f400819a710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 --2- 172.21.15.3:0/3822324174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40080742f0 0x7f400819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 >> 172.21.15.3:0/3822324174 conn(0x7f40080fd470 msgr2=0x7f40080fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 shutdown_connections 2024-08-01T04:17:08.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:08.662+0000 7f4007e00640 1 -- 172.21.15.3:0/3822324174 wait complete. 2024-08-01T04:17:09.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:09 smithi163 ceph-mon[37383]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-01T04:17:09.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:09 smithi003 ceph-mon[32262]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-01T04:17:09.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:09 smithi160 ceph-mon[37224]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-01T04:17:09.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:17:10.030 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.672+0000 7f86c3200640 1 -- 172.21.15.3:0/1996587349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c41004b0 msgr2=0x7f86c4100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.672+0000 7f86c3200640 1 --2- 172.21.15.3:0/1996587349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c41004b0 0x7f86c4100890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009860 tx=0x7f86b402a340 comp rx=0 tx=0).stop 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.672+0000 7f86c3200640 1 -- 172.21.15.3:0/1996587349 shutdown_connections 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.673+0000 7f86c3200640 1 --2- 172.21.15.3:0/1996587349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c410e060 0x7f86c4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.673+0000 7f86c3200640 1 --2- 172.21.15.3:0/1996587349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c4100e60 0x7f86c410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.673+0000 7f86c3200640 1 --2- 172.21.15.3:0/1996587349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c41004b0 0x7f86c4100890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.673+0000 7f86c3200640 1 -- 172.21.15.3:0/1996587349 >> 172.21.15.3:0/1996587349 conn(0x7f86c40fc340 msgr2=0x7f86c40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.673+0000 7f86c3200640 1 -- 172.21.15.3:0/1996587349 shutdown_connections 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.673+0000 7f86c3200640 1 -- 172.21.15.3:0/1996587349 wait complete. 2024-08-01T04:17:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 Processor -- start 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 -- start start 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 0x7f86c419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c4100e60 0x7f86c419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c410e060 0x7f86c419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4113d10 con 0x7f86c41004b0 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4113b90 con 0x7f86c410e060 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.674+0000 7f86c3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4113e90 con 0x7f86c4100e60 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 0x7f86c419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 0x7f86c419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37126/0 (socket says 172.21.15.3:37126) 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c2800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c410e060 0x7f86c419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 -- 172.21.15.3:0/1857264737 learned_addr learned my addr 172.21.15.3:0/1857264737 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1400640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c4100e60 0x7f86c419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 -- 172.21.15.3:0/1857264737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c4100e60 msgr2=0x7f86c419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c4100e60 0x7f86c419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 -- 172.21.15.3:0/1857264737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c410e060 msgr2=0x7f86c419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c410e060 0x7f86c419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1e00640 1 -- 172.21.15.3:0/1857264737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c419f1e0 con 0x7f86c41004b0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c1400640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c4100e60 0x7f86c419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.675+0000 7f86c2800640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c410e060 0x7f86c419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.676+0000 7f86c1e00640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 0x7f86c419a230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f86b4035040 tx=0x7f86b4033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.676+0000 7f86baa00640 1 -- 172.21.15.3:0/1857264737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b403f020 con 0x7f86c41004b0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.676+0000 7f86baa00640 1 -- 172.21.15.3:0/1857264737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b4032770 con 0x7f86c41004b0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.676+0000 7f86baa00640 1 -- 172.21.15.3:0/1857264737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4037040 con 0x7f86c41004b0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.676+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c419f470 con 0x7f86c41004b0 2024-08-01T04:17:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.676+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c41a6db0 con 0x7f86c41004b0 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.680+0000 7f86cbc14740 1 Processor -- start 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.680+0000 7f86cbc14740 1 -- start start 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.680+0000 7f86baa00640 1 -- 172.21.15.3:0/1857264737 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f86b4042020 con 0x7f86c41004b0 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec4325fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec4326a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec4326ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86cbc14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec431c4bf0 con 0x55ec4325f6e0 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86cbc14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ec4315cb20 con 0x55ec4326a8c0 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86cbc14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec431c0bb0 con 0x55ec43260000 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86c1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec4326a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86c2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec4325fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86c1e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec4326a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51984/0 (socket says 172.21.15.3:51984) 2024-08-01T04:17:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86c1400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec4326ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86c2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec4325fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37138/0 (socket says 172.21.15.3:37138) 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.681+0000 7f86c1e00640 1 -- 172.21.15.3:0/2595773113 learned_addr learned my addr 172.21.15.3:0/2595773113 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86c1400640 1 -- 172.21.15.3:0/2595773113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 msgr2=0x55ec4326a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86c1400640 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec4326a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86c1400640 1 -- 172.21.15.3:0/2595773113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 msgr2=0x55ec4325fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86c1400640 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec4325fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86c1400640 1 -- 172.21.15.3:0/2595773113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec431637f0 con 0x55ec4326a8c0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86c1400640 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec4326ccb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005fd0 tx=0x7f86b00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78797f24ff307384 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86baa00640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f86a80a2ea0 0x7f86a80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.682+0000 7f86af400640 1 -- 172.21.15.3:0/2595773113 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0012450 con 0x55ec4326a8c0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.683+0000 7f86af400640 1 -- 172.21.15.3:0/2595773113 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b0012ad0 con 0x55ec4326a8c0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.683+0000 7f86af400640 1 -- 172.21.15.3:0/2595773113 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b000fba0 con 0x55ec4326a8c0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.683+0000 7f86baa00640 1 -- 172.21.15.3:0/1857264737 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f86b40ef550 con 0x7f86c41004b0 2024-08-01T04:17:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.683+0000 7f86c2800640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f86a80a2ea0 0x7f86a80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.683+0000 7f86cbc14740 1 -- 172.21.15.3:0/2595773113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 msgr2=0x55ec4326ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.683+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec4326ccb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005fd0 tx=0x7f86b00117b0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86c2800640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f86a80a2ea0 0x7f86a80a5360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86c4102690 tx=0x7f86bc009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 -- 172.21.15.3:0/2595773113 shutdown_connections 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec4326ccb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec4326a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2595773113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec4325fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 -- 172.21.15.3:0/2595773113 >> 172.21.15.3:0/2595773113 conn(0x55ec4325bc70 msgr2=0x55ec4325c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 -- 172.21.15.3:0/2595773113 shutdown_connections 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 -- 172.21.15.3:0/2595773113 wait complete. 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.684+0000 7f86cbc14740 1 -- start start 2024-08-01T04:17:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 Processor -- start 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4325f6e0 0x55ec432676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec43260000 0x55ec43267bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 0x55ec43268130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec4326d490 con 0x55ec4326a8c0 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ec4317fcf0 con 0x55ec43260000 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55ec4326d610 con 0x55ec4325f6e0 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.685+0000 7f86cbc14740 10 client.0 _ll_register_callbacks cb 0x55ec4316be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 0x55ec43268130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 0x55ec43268130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37144/0 (socket says 172.21.15.3:37144) 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 -- 172.21.15.3:0/2419684561 learned_addr learned my addr 172.21.15.3:0/2419684561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c1400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec43260000 0x55ec43267bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c1e00640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4325f6e0 0x55ec432676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 -- 172.21.15.3:0/2419684561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4325f6e0 msgr2=0x55ec432676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4325f6e0 0x55ec432676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 -- 172.21.15.3:0/2419684561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec43260000 msgr2=0x55ec43267bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec43260000 0x55ec43267bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 -- 172.21.15.3:0/2419684561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec42f363c0 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c1400640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec43260000 0x55ec43267bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86c2800640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 0x55ec43268130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00fef0 tx=0x7f86bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982094a0a57bc121 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.686+0000 7f86a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.687+0000 7f86a3e00640 1 -- 172.21.15.3:0/2419684561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc02e060 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.687+0000 7f86a3e00640 1 -- 172.21.15.3:0/2419684561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f86bc018030 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.687+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b0012ad0 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.687+0000 7f86a3e00640 1 -- 172.21.15.3:0/2419684561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86bc01e9e0 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.687+0000 7f86cbc14740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.687+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec432790a0 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86a3e00640 1 -- 172.21.15.3:0/2419684561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b0012ad0 con 0x55ec4326a8c0 2024-08-01T04:17:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86a3e00640 1 -- 172.21.15.3:0/2419684561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f86bc0313f0 con 0x55ec4326a8c0 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86cbc14740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86cbc14740 10 client.15276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86cbc14740 20 client.15276 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec4327a560 0x55ec4327c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86cbc14740 4 client.15276 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec431d2160 con 0x55ec4327a560 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.688+0000 7f86c1e00640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec4327a560 0x55ec4327c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.689+0000 7f86c1e00640 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec4327a560 0x55ec4327c9b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb3365d0c9f8e329 server_cookie=d55b9ec78ebf6782 in_seq=0 out_seq=0 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.689+0000 7f86a3e00640 10 client.15276 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86a3e00640 1 -- 172.21.15.3:0/2419684561 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec431d2160 con 0x55ec4327a560 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86a3e00640 10 client.15276 handle_command_reply: tid=0 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 client.15276 shutdown 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 msgr2=0x55ec43268130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 0x55ec43268130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00fef0 tx=0x7f86bc004ab0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 shutdown_connections 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec4327a560 0x55ec4327c9b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4326a8c0 0x55ec43268130 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec43260000 0x55ec43267bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2419684561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4325f6e0 0x55ec432676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86a3e00640 0 client.15276 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 >> 172.21.15.3:0/2419684561 conn(0x55ec4325bc70 msgr2=0x55ec432619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 shutdown_connections 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.690+0000 7f86cbc14740 1 -- 172.21.15.3:0/2419684561 wait complete. 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.691+0000 7f86cbc14740 20 client.15276 trim_cache size 0 max 16384 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 Processor -- start 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 -- start start 2024-08-01T04:17:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec43256910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec43282280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec432827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41b2110 con 0x55ec4325f6e0 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41af9e0 con 0x55ec4326a8c0 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.706+0000 7f86cbc14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41ad830 con 0x55ec43260000 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c2800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec432827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec43256910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c2800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec432827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51564/0 (socket says 172.21.15.3:51564) 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c2800640 1 -- 172.21.15.3:0/2271901485 learned_addr learned my addr 172.21.15.3:0/2271901485 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec43282280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c2800640 1 -- 172.21.15.3:0/2271901485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 msgr2=0x55ec43282280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.707+0000 7f86c2800640 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec43282280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86c2800640 1 -- 172.21.15.3:0/2271901485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 msgr2=0x55ec43256910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86c2800640 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec43256910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86c2800640 1 -- 172.21.15.3:0/2271901485 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4327d030 con 0x55ec4326a8c0 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86c1400640 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec43282280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86c2800640 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec432827c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f86bc019040 tx=0x7f86bc031c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43328085445e80f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86aea00640 1 -- 172.21.15.3:0/2271901485 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc005bd0 con 0x55ec4326a8c0 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86aea00640 1 -- 172.21.15.3:0/2271901485 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc03f070 con 0x55ec4326a8c0 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.708+0000 7f86aea00640 1 -- 172.21.15.3:0/2271901485 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc004040 con 0x55ec4326a8c0 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 -- 172.21.15.3:0/2271901485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 msgr2=0x55ec432827c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec432827c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f86bc019040 tx=0x7f86bc031c90 comp rx=0 tx=0).stop 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 -- 172.21.15.3:0/2271901485 shutdown_connections 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4326a8c0 0x55ec432827c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec43260000 0x55ec43282280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2271901485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec4325f6e0 0x55ec43256910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 -- 172.21.15.3:0/2271901485 >> 172.21.15.3:0/2271901485 conn(0x55ec4325bc70 msgr2=0x55ec432667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 -- 172.21.15.3:0/2271901485 shutdown_connections 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.709+0000 7f86cbc14740 1 -- 172.21.15.3:0/2271901485 wait complete. 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.710+0000 7f86cbc14740 1 -- start start 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.710+0000 7f86cbc14740 1 Processor -- start 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.710+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4325f6e0 0x55ec43272d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.710+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 0x55ec43264170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86cbc14740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4326a8c0 0x55ec432646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86cbc14740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41adea0 con 0x55ec43260000 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86cbc14740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41af310 con 0x55ec4325f6e0 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86cbc14740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41ae570 con 0x55ec4326a8c0 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86cbc14740 10 client.0 _ll_register_callbacks cb 0x7f86bc00fd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 0x55ec43264170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 0x55ec43264170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37172/0 (socket says 172.21.15.3:37172) 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 -- 172.21.15.3:0/2126706612 learned_addr learned my addr 172.21.15.3:0/2126706612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1e00640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4325f6e0 0x55ec43272d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c2800640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4326a8c0 0x55ec432646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 -- 172.21.15.3:0/2126706612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4326a8c0 msgr2=0x55ec432646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4326a8c0 0x55ec432646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 -- 172.21.15.3:0/2126706612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4325f6e0 msgr2=0x55ec43272d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4325f6e0 0x55ec43272d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.711+0000 7f86c1400640 1 -- 172.21.15.3:0/2126706612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec43264bf0 con 0x55ec43260000 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86c1400640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 0x55ec43264170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f86b001b000 tx=0x7f86b000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eda0e26c047d6a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86a3e00640 1 -- 172.21.15.3:0/2126706612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0005450 con 0x55ec43260000 2024-08-01T04:17:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86bc03f070 con 0x55ec43260000 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86a3e00640 1 -- 172.21.15.3:0/2126706612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f86b000f870 con 0x55ec43260000 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86a3e00640 1 -- 172.21.15.3:0/2126706612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86b00055f0 con 0x55ec43260000 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86cbc14740 10 client.15282 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.712+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ec432715c0 con 0x55ec43260000 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86a3e00640 1 -- 172.21.15.3:0/2126706612 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc03f070 con 0x55ec43260000 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86a3e00640 1 -- 172.21.15.3:0/2126706612 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f86b00046f0 con 0x55ec43260000 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86cbc14740 10 client.15282 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86cbc14740 10 client.15282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86cbc14740 20 client.15282 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec432e53f0 0x55ec432e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86cbc14740 4 client.15282 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec431cc670 con 0x55ec432e53f0 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.713+0000 7f86c1e00640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec432e53f0 0x55ec432e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86c1e00640 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec432e53f0 0x55ec432e77e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c0d53d90dc2f836 server_cookie=3ec1618b18d1587f in_seq=0 out_seq=0 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86a3e00640 10 client.15282 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86a3e00640 1 -- 172.21.15.3:0/2126706612 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec431cc670 con 0x55ec432e53f0 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86a3e00640 10 client.15282 handle_command_reply: tid=0 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 client.15282 shutdown 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 msgr2=0x55ec43264170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 0x55ec43264170 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f86b001b000 tx=0x7f86b000f670 comp rx=0 tx=0).stop 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 shutdown_connections 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55ec432e53f0 0x55ec432e77e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ec4326a8c0 0x55ec432646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.715+0000 7f86a3e00640 0 client.15282 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.714+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec43260000 0x55ec43264170 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.715+0000 7f86cbc14740 1 --2- 172.21.15.3:0/2126706612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ec4325f6e0 0x55ec43272d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.715+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 >> 172.21.15.3:0/2126706612 conn(0x55ec4325bc70 msgr2=0x55ec43268f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.715+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 shutdown_connections 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.715+0000 7f86cbc14740 1 -- 172.21.15.3:0/2126706612 wait complete. 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.715+0000 7f86cbc14740 20 client.15282 trim_cache size 0 max 16384 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f86a80a2ea0 msgr2=0x7f86a80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f86a80a2ea0 0x7f86a80a5360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86c4102690 tx=0x7f86bc009500 comp rx=0 tx=0).stop 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 msgr2=0x7f86c419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 0x7f86c419a230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f86b4035040 tx=0x7f86b4033d90 comp rx=0 tx=0).stop 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 shutdown_connections 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86c410e060 0x7f86c419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f86a80a2ea0 0x7f86a80a5360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86c4100e60 0x7f86c419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 --2- 172.21.15.3:0/1857264737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86c41004b0 0x7f86c419a230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 >> 172.21.15.3:0/1857264737 conn(0x7f86c40fc340 msgr2=0x7f86c40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 shutdown_connections 2024-08-01T04:17:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:10.716+0000 7f86c3200640 1 -- 172.21.15.3:0/1857264737 wait complete. 2024-08-01T04:17:11.283 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:11 smithi003 ceph-mon[32262]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:11.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:11 smithi163 ceph-mon[37383]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:11.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:11 smithi160 ceph-mon[37224]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-08-01T04:17:11.684 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:17:12.133 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:12.562 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-01T04:17:12.562 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.4 2024-08-01T04:17:12.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:17:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 -- 172.21.15.3:0/2151400622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1016f0 msgr2=0x7f94dc101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2151400622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1016f0 0x7f94dc101b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009860 tx=0x7f94c802a340 comp rx=0 tx=0).stop 2024-08-01T04:17:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 -- 172.21.15.3:0/2151400622 shutdown_connections 2024-08-01T04:17:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2151400622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc102090 0x7f94dc10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2151400622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1016f0 0x7f94dc101b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2151400622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94dc1076f0 0x7f94dc107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 -- 172.21.15.3:0/2151400622 >> 172.21.15.3:0/2151400622 conn(0x7f94dc0fd470 msgr2=0x7f94dc0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 -- 172.21.15.3:0/2151400622 shutdown_connections 2024-08-01T04:17:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.781+0000 7f94dbe00640 1 -- 172.21.15.3:0/2151400622 wait complete. 2024-08-01T04:17:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 Processor -- start 2024-08-01T04:17:12.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 -- start start 2024-08-01T04:17:12.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 0x7f94dc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94dc102090 0x7f94dc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1076f0 0x7f94dc19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc111ab0 con 0x7f94dc102090 2024-08-01T04:17:12.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc111930 con 0x7f94dc1076f0 2024-08-01T04:17:12.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.783+0000 7f94dbe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc111c30 con 0x7f94dc1016f0 2024-08-01T04:17:12.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 0x7f94dc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 0x7f94dc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55738/0 (socket says 172.21.15.3:55738) 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94da000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94dc102090 0x7f94dc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 -- 172.21.15.3:0/2057834746 learned_addr learned my addr 172.21.15.3:0/2057834746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94db400640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1076f0 0x7f94dc19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 -- 172.21.15.3:0/2057834746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1076f0 msgr2=0x7f94dc19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1076f0 0x7f94dc19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 -- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94dc102090 msgr2=0x7f94dc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94dc102090 0x7f94dc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.784+0000 7f94daa00640 1 -- 172.21.15.3:0/2057834746 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc19cf60 con 0x7f94dc1016f0 2024-08-01T04:17:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.785+0000 7f94daa00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 0x7f94dc197fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00ea60 tx=0x7f94cc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.785+0000 7f94d3400640 1 -- 172.21.15.3:0/2057834746 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc00cde0 con 0x7f94dc1016f0 2024-08-01T04:17:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.785+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc19d250 con 0x7f94dc1016f0 2024-08-01T04:17:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.785+0000 7f94d3400640 1 -- 172.21.15.3:0/2057834746 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94cc0035f0 con 0x7f94dc1016f0 2024-08-01T04:17:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.785+0000 7f94d3400640 1 -- 172.21.15.3:0/2057834746 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc010670 con 0x7f94dc1016f0 2024-08-01T04:17:12.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.786+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc074150 con 0x7f94dc1016f0 2024-08-01T04:17:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.789+0000 7f94e4404740 1 Processor -- start 2024-08-01T04:17:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.789+0000 7f94e4404740 1 -- start start 2024-08-01T04:17:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.789+0000 7f94d3400640 1 -- 172.21.15.3:0/2057834746 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f94cc0037a0 con 0x7f94dc1016f0 2024-08-01T04:17:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.789+0000 7f94e4404740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6eef0 0x55e76fc6f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.789+0000 7f94e4404740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc79b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94e4404740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 0x55e76fc7c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94e4404740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e76fbcbbf0 con 0x55e76fc6eef0 2024-08-01T04:17:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94e4404740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e76fb63b20 con 0x55e76fc6f810 2024-08-01T04:17:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94e4404740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e76fbc7bb0 con 0x55e76fc7a0d0 2024-08-01T04:17:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94db400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6eef0 0x55e76fc6f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94da000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 0x55e76fc7c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94daa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc79b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94da000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 0x55e76fc7c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55754/0 (socket says 172.21.15.3:55754) 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.790+0000 7f94da000640 1 -- 172.21.15.3:0/3785562361 learned_addr learned my addr 172.21.15.3:0/3785562361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94da000640 1 -- 172.21.15.3:0/3785562361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 msgr2=0x55e76fc79b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94da000640 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc79b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94da000640 1 -- 172.21.15.3:0/3785562361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6eef0 msgr2=0x55e76fc6f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94da000640 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6eef0 0x55e76fc6f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94da000640 1 -- 172.21.15.3:0/3785562361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76fb6a7f0 con 0x55e76fc7a0d0 2024-08-01T04:17:12.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94da000640 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 0x55e76fc7c4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009780 tx=0x7f94c8009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79a4e4528a83865 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94c7e00640 1 -- 172.21.15.3:0/3785562361 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8040020 con 0x55e76fc7a0d0 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94c7e00640 1 -- 172.21.15.3:0/3785562361 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c8030e40 con 0x55e76fc7a0d0 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94c7e00640 1 -- 172.21.15.3:0/3785562361 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8037040 con 0x55e76fc7a0d0 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.791+0000 7f94d3400640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f94c00a2f20 0x7f94c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.792+0000 7f94d3400640 1 -- 172.21.15.3:0/2057834746 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f94cc0cfa60 con 0x7f94dc1016f0 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.792+0000 7f94e4404740 1 -- 172.21.15.3:0/3785562361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 msgr2=0x55e76fc7c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.792+0000 7f94e4404740 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 0x55e76fc7c4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009780 tx=0x7f94c8009510 comp rx=0 tx=0).stop 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.792+0000 7f94db400640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f94c00a2f20 0x7f94c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 -- 172.21.15.3:0/3785562361 shutdown_connections 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc7a0d0 0x55e76fc7c4c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc79b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 --2- 172.21.15.3:0/3785562361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6eef0 0x55e76fc6f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 -- 172.21.15.3:0/3785562361 >> 172.21.15.3:0/3785562361 conn(0x55e76fc6b480 msgr2=0x55e76fc6b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 -- 172.21.15.3:0/3785562361 shutdown_connections 2024-08-01T04:17:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94e4404740 1 -- 172.21.15.3:0/3785562361 wait complete. 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.793+0000 7f94db400640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f94c00a2f20 0x7f94c00a53e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94d4005fd0 tx=0x7f94d4011040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.794+0000 7f94e4404740 1 -- start start 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.794+0000 7f94e4404740 1 Processor -- start 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.794+0000 7f94e4404740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc76f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.794+0000 7f94e4404740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fc774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.794+0000 7f94e4404740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fc77a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94e4404740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e76fc7cca0 con 0x55e76fc6f810 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94e4404740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e76fb86cf0 con 0x55e76fc7a0d0 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94e4404740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e76fc7ce20 con 0x55e76fc6eef0 2024-08-01T04:17:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94e4404740 10 client.0 _ll_register_callbacks cb 0x55e76fb72e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94daa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc76f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94da000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fc774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94daa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc76f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55758/0 (socket says 172.21.15.3:55758) 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94daa00640 1 -- 172.21.15.3:0/1282440837 learned_addr learned my addr 172.21.15.3:0/1282440837 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.795+0000 7f94db400640 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fc77a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94daa00640 1 -- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 msgr2=0x55e76fc77a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94daa00640 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fc77a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94daa00640 1 -- 172.21.15.3:0/1282440837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 msgr2=0x55e76fc774c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94daa00640 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fc774c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94daa00640 1 -- 172.21.15.3:0/1282440837 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76f93d3c0 con 0x55e76fc6eef0 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94daa00640 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc76f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f94cc09ba00 tx=0x7f94cc0d3140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72db4cefebc4cb6f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94c4c00640 1 -- 172.21.15.3:0/1282440837 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0d9040 con 0x55e76fc6eef0 2024-08-01T04:17:12.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c8030e40 con 0x55e76fc6eef0 2024-08-01T04:17:12.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.796+0000 7f94c4c00640 1 -- 172.21.15.3:0/1282440837 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f94cc09b150 con 0x55e76fc6eef0 2024-08-01T04:17:12.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.797+0000 7f94c4c00640 1 -- 172.21.15.3:0/1282440837 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94cc09bad0 con 0x55e76fc6eef0 2024-08-01T04:17:12.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.797+0000 7f94e4404740 10 client.25408 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:12.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.797+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e76fc889d0 con 0x55e76fc6eef0 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.797+0000 7f94c4c00640 1 -- 172.21.15.3:0/1282440837 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c8030e40 con 0x55e76fc6eef0 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.797+0000 7f94c4c00640 1 -- 172.21.15.3:0/1282440837 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f94cc097a50 con 0x55e76fc6eef0 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94e4404740 10 client.25408 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94e4404740 10 client.25408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94e4404740 20 client.25408 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94e4404740 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc89de0 0x55e76fc8c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94e4404740 4 client.25408 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e76fbd9160 con 0x55e76fc89de0 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.798+0000 7f94da000640 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc89de0 0x55e76fc8c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.799+0000 7f94da000640 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc89de0 0x55e76fc8c250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887f7ea132a95518 server_cookie=34d8869a9378b1aa in_seq=0 out_seq=0 2024-08-01T04:17:12.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.799+0000 7f94c4c00640 10 client.25408 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:12.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.799+0000 7f94c4c00640 1 -- 172.21.15.3:0/1282440837 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e76fbd9160 con 0x55e76fc89de0 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.799+0000 7f94c4c00640 10 client.25408 handle_command_reply: tid=0 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 client.25408 shutdown 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 msgr2=0x55e76fc76f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc76f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f94cc09ba00 tx=0x7f94cc0d3140 comp rx=0 tx=0).stop 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 shutdown_connections 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc89de0 0x55e76fc8c250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fc77a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fc774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 --2- 172.21.15.3:0/1282440837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc76f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 >> 172.21.15.3:0/1282440837 conn(0x55e76fc6b480 msgr2=0x55e76fc711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94c4c00640 0 client.25408 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.800+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 shutdown_connections 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.801+0000 7f94e4404740 1 -- 172.21.15.3:0/1282440837 wait complete. 2024-08-01T04:17:12.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.801+0000 7f94e4404740 20 client.25408 trim_cache size 0 max 16384 2024-08-01T04:17:12.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.815+0000 7f94e4404740 1 Processor -- start 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.815+0000 7f94e4404740 1 -- start start 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.815+0000 7f94e4404740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.815+0000 7f94e4404740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.815+0000 7f94e4404740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 0x55e76fc92080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.815+0000 7f94e4404740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1adb90 con 0x55e76fc7a0d0 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94e4404740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1ab460 con 0x55e76fc6f810 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94e4404740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1a92b0 con 0x55e76fc6eef0 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 0x55e76fc92080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 0x55e76fc92080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41996/0 (socket says 172.21.15.3:41996) 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 -- 172.21.15.3:0/3276618837 learned_addr learned my addr 172.21.15.3:0/3276618837 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:12.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94daa00640 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 -- 172.21.15.3:0/3276618837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 msgr2=0x55e76fc71950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc71950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 -- 172.21.15.3:0/3276618837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 msgr2=0x55e76fc91b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:17:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.816+0000 7f94db400640 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc91b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.817+0000 7f94db400640 1 -- 172.21.15.3:0/3276618837 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76fc8c5c0 con 0x55e76fc7a0d0 2024-08-01T04:17:12.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.817+0000 7f94db400640 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 0x55e76fc92080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94d401b900 tx=0x7f94d401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cba9f290d7e635 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.818+0000 7f94c7400640 1 -- 172.21.15.3:0/3276618837 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d401ac70 con 0x55e76fc7a0d0 2024-08-01T04:17:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.818+0000 7f94c7400640 1 -- 172.21.15.3:0/3276618837 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d402c070 con 0x55e76fc7a0d0 2024-08-01T04:17:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.818+0000 7f94c7400640 1 -- 172.21.15.3:0/3276618837 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40185f0 con 0x55e76fc7a0d0 2024-08-01T04:17:12.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.818+0000 7f94e4404740 1 -- 172.21.15.3:0/3276618837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 msgr2=0x55e76fc92080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.818+0000 7f94e4404740 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 0x55e76fc92080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94d401b900 tx=0x7f94d401a000 comp rx=0 tx=0).stop 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 -- 172.21.15.3:0/3276618837 shutdown_connections 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc7a0d0 0x55e76fc92080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc6f810 0x55e76fc91b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 --2- 172.21.15.3:0/3276618837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fc71950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 -- 172.21.15.3:0/3276618837 >> 172.21.15.3:0/3276618837 conn(0x55e76fc6b480 msgr2=0x55e76fc722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 -- 172.21.15.3:0/3276618837 shutdown_connections 2024-08-01T04:17:12.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.819+0000 7f94e4404740 1 -- 172.21.15.3:0/3276618837 wait complete. 2024-08-01T04:17:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.820+0000 7f94e4404740 1 -- start start 2024-08-01T04:17:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.820+0000 7f94e4404740 1 Processor -- start 2024-08-01T04:17:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fcece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fced3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fced930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1a9920 con 0x55e76fc6f810 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1aad90 con 0x55e76fc7a0d0 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1a9ff0 con 0x55e76fc6eef0 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94e4404740 10 client.0 _ll_register_callbacks cb 0x7f94cc0d3220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94da000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fced3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94daa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fcece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94da000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fced3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42006/0 (socket says 172.21.15.3:42006) 2024-08-01T04:17:12.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.821+0000 7f94da000640 1 -- 172.21.15.3:0/521544517 learned_addr learned my addr 172.21.15.3:0/521544517 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94da000640 1 -- 172.21.15.3:0/521544517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 msgr2=0x55e76fcece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94da000640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fcece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94da000640 1 -- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 msgr2=0x55e76fced930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:12.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94db400640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fced930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94da000640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fced930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94da000640 1 -- 172.21.15.3:0/521544517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76fc81320 con 0x55e76fc6f810 2024-08-01T04:17:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94daa00640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fcece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94db400640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fced930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94da000640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fced3d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94c802d020 tx=0x7f94c80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c931387a58242020 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:12.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94c4c00640 1 -- 172.21.15.3:0/521544517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8040040 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94c4c00640 1 -- 172.21.15.3:0/521544517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f94c8030580 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.822+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d402c070 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94c4c00640 1 -- 172.21.15.3:0/521544517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94c8037040 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 10 client.15300 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e76fc82650 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94c4c00640 1 -- 172.21.15.3:0/521544517 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d402c070 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94c4c00640 1 -- 172.21.15.3:0/521544517 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f94c8003800 con 0x55e76fc6f810 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 20 client.15300 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:12.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc829e0 0x55e76fc68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 4 client.15300 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:17:12.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.823+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e76fbd3670 con 0x55e76fc829e0 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.824+0000 7f94daa00640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc829e0 0x55e76fc68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94c4c00640 10 client.15300 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94daa00640 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc829e0 0x55e76fc68610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db48e60830cc4961 server_cookie=77114dadc5c240ad in_seq=0 out_seq=0 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94c4c00640 1 -- 172.21.15.3:0/521544517 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e76fbd3670 con 0x55e76fc829e0 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94c4c00640 10 client.15300 handle_command_reply: tid=0 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 client.15300 shutdown 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 msgr2=0x55e76fced3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fced3d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94c802d020 tx=0x7f94c80035f0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 shutdown_connections 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e76fc829e0 0x55e76fc68610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e76fc7a0d0 0x55e76fced930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e76fc6f810 0x55e76fced3d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 --2- 172.21.15.3:0/521544517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e76fc6eef0 0x55e76fcece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.825+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 >> 172.21.15.3:0/521544517 conn(0x55e76fc6b480 msgr2=0x55e76fc61480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.826+0000 7f94c4c00640 0 client.15300 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.826+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 shutdown_connections 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.826+0000 7f94e4404740 1 -- 172.21.15.3:0/521544517 wait complete. 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.826+0000 7f94e4404740 20 client.15300 trim_cache size 0 max 16384 2024-08-01T04:17:12.829 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f94c00a2f20 msgr2=0x7f94c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f94c00a2f20 0x7f94c00a53e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94d4005fd0 tx=0x7f94d4011040 comp rx=0 tx=0).stop 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 msgr2=0x7f94dc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 0x7f94dc197fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00ea60 tx=0x7f94cc00ef20 comp rx=0 tx=0).stop 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 shutdown_connections 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94dc1076f0 0x7f94dc19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f94dc102090 0x7f94dc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f94c00a2f20 0x7f94c00a53e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 --2- 172.21.15.3:0/2057834746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94dc1016f0 0x7f94dc197fb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:12.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 >> 172.21.15.3:0/2057834746 conn(0x7f94dc0fd470 msgr2=0x7f94dc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:12.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 shutdown_connections 2024-08-01T04:17:12.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:12.827+0000 7f94dbe00640 1 -- 172.21.15.3:0/2057834746 wait complete. 2024-08-01T04:17:13.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:13 smithi163 ceph-mon[37383]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-01T04:17:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:13 smithi003 ceph-mon[32262]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-01T04:17:13.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:13 smithi160 ceph-mon[37224]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-01T04:17:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.365+0000 7f1d80a00640 1 -- 172.21.15.3:0/114085718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d740319b0 con 0x7f1d840742c0 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.365+0000 7f1d83200640 1 -- 172.21.15.3:0/114085718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d840742c0 msgr2=0x7f1d840746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.365+0000 7f1d83200640 1 --2- 172.21.15.3:0/114085718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d840742c0 0x7f1d840746a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009900 tx=0x7f1d7402a480 comp rx=0 tx=0).stop 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 -- 172.21.15.3:0/114085718 shutdown_connections 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 --2- 172.21.15.3:0/114085718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d84112710 0x7f1d84114b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 --2- 172.21.15.3:0/114085718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d84074be0 0x7f1d841121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 --2- 172.21.15.3:0/114085718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d840742c0 0x7f1d840746a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 -- 172.21.15.3:0/114085718 >> 172.21.15.3:0/114085718 conn(0x7f1d840fd440 msgr2=0x7f1d840ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:14.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 -- 172.21.15.3:0/114085718 shutdown_connections 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.366+0000 7f1d83200640 1 -- 172.21.15.3:0/114085718 wait complete. 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.367+0000 7f1d83200640 1 Processor -- start 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.367+0000 7f1d83200640 1 -- start start 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d83200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 0x7f1d8419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d83200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d84074be0 0x7f1d8419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d83200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d84112710 0x7f1d841a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d83200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841180d0 con 0x7f1d84074be0 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d83200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84117f50 con 0x7f1d840742c0 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d83200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84118250 con 0x7f1d84112710 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d81400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d84074be0 0x7f1d8419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:14.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d81e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 0x7f1d8419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d81e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 0x7f1d8419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39314/0 (socket says 172.21.15.3:39314) 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d81e00640 1 -- 172.21.15.3:0/1899175512 learned_addr learned my addr 172.21.15.3:0/1899175512 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.368+0000 7f1d82800640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d84112710 0x7f1d841a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.369+0000 7f1d81e00640 1 -- 172.21.15.3:0/1899175512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d84112710 msgr2=0x7f1d841a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.369+0000 7f1d81e00640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d84112710 0x7f1d841a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.369+0000 7f1d81e00640 1 -- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d84074be0 msgr2=0x7f1d8419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.369+0000 7f1d81e00640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d84074be0 0x7f1d8419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.369+0000 7f1d81e00640 1 -- 172.21.15.3:0/1899175512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d841a3640 con 0x7f1d840742c0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.369+0000 7f1d81e00640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 0x7f1d8419e690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d74006010 tx=0x7f1d74004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.370+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7403f030 con 0x7f1d840742c0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.370+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d74004440 con 0x7f1d840742c0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.370+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d841a38d0 con 0x7f1d840742c0 2024-08-01T04:17:14.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.370+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74035070 con 0x7f1d840742c0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.370+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d841ab210 con 0x7f1d840742c0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.371+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d840746a0 con 0x7f1d840742c0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.373+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f1d74042030 con 0x7f1d840742c0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.376+0000 7f1d7aa00640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d680a2f20 0x7f1d680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.376+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1d740efdf0 con 0x7f1d840742c0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.376+0000 7f1d81400640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d680a2f20 0x7f1d680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.377+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d740bc200 con 0x7f1d840742c0 2024-08-01T04:17:14.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.377+0000 7f1d81400640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d680a2f20 0x7f1d680a53e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d840ffcd0 tx=0x7f1d70005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:14.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.619+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d8419f9f0 con 0x7f1d840742c0 2024-08-01T04:17:14.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.620+0000 7f1d7aa00640 1 -- 172.21.15.3:0/1899175512 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7f1d740c0500 con 0x7f1d840742c0 2024-08-01T04:17:14.622 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:17:14.623 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:17:14.623 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:17:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.623+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d680a2f20 msgr2=0x7f1d680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.623+0000 7f1d83200640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d680a2f20 0x7f1d680a53e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d840ffcd0 tx=0x7f1d70005f90 comp rx=0 tx=0).stop 2024-08-01T04:17:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 msgr2=0x7f1d8419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:14.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 0x7f1d8419e690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d74006010 tx=0x7f1d74004040 comp rx=0 tx=0).stop 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 shutdown_connections 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f1d680a2f20 0x7f1d680a53e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d84112710 0x7f1d841a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d84074be0 0x7f1d8419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 --2- 172.21.15.3:0/1899175512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1d840742c0 0x7f1d8419e690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 >> 172.21.15.3:0/1899175512 conn(0x7f1d840fd440 msgr2=0x7f1d840ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.624+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 shutdown_connections 2024-08-01T04:17:14.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:14.625+0000 7f1d83200640 1 -- 172.21.15.3:0/1899175512 wait complete. 2024-08-01T04:17:14.724 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:17:14.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:17:15.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:15 smithi163 ceph-mon[37383]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-01T04:17:15.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:15 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1899175512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:15.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:15 smithi003 ceph-mon[32262]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-01T04:17:15.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:15 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1899175512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:15.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:15 smithi160 ceph-mon[37224]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-01T04:17:15.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:15 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1899175512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:15.476 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db / --timeout 90 --expiration 144.4 --await-for 45.0' 2024-08-01T04:17:15.476 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db / --timeout 90 --expiration 144.4 --await-for 45.0 2024-08-01T04:17:15.590 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 -- 172.21.15.3:0/2377647461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0109b70 msgr2=0x7ff0b0109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/2377647461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0109b70 0x7ff0b0109f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009890 tx=0x7ff0a402a360 comp rx=0 tx=0).stop 2024-08-01T04:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 -- 172.21.15.3:0/2377647461 shutdown_connections 2024-08-01T04:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/2377647461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b0104480 0x7ff0b01109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/2377647461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0b0103ae0 0x7ff0b0103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/2377647461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0109b70 0x7ff0b0109f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.286+0000 7ff0b7e00640 1 -- 172.21.15.3:0/2377647461 >> 172.21.15.3:0/2377647461 conn(0x7ff0b00ff780 msgr2=0x7ff0b0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:16.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.287+0000 7ff0b7e00640 1 -- 172.21.15.3:0/2377647461 shutdown_connections 2024-08-01T04:17:16.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.287+0000 7ff0b7e00640 1 -- 172.21.15.3:0/2377647461 wait complete. 2024-08-01T04:17:16.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 Processor -- start 2024-08-01T04:17:16.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 -- start start 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b0103ae0 0x7ff0b019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 0x7ff0b019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0b0109b70 0x7ff0b019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0113ee0 con 0x7ff0b0103ae0 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0113d60 con 0x7ff0b0109b70 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.288+0000 7ff0b7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b0114060 con 0x7ff0b0104480 2024-08-01T04:17:16.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 0x7ff0b019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 0x7ff0b019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55782/0 (socket says 172.21.15.3:55782) 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 -- 172.21.15.3:0/1220397216 learned_addr learned my addr 172.21.15.3:0/1220397216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0b0109b70 0x7ff0b019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 -- 172.21.15.3:0/1220397216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0b0109b70 msgr2=0x7ff0b019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0b0109b70 0x7ff0b019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 -- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b0103ae0 msgr2=0x7ff0b019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b0103ae0 0x7ff0b019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.289+0000 7ff0b4a00640 1 -- 172.21.15.3:0/1220397216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b019f370 con 0x7ff0b0104480 2024-08-01T04:17:16.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.290+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 0x7ff0b019a900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e9f0 tx=0x7ff09c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:16.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.290+0000 7ff0ae000640 1 -- 172.21.15.3:0/1220397216 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00cd80 con 0x7ff0b0104480 2024-08-01T04:17:16.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.290+0000 7ff0ae000640 1 -- 172.21.15.3:0/1220397216 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c010910 con 0x7ff0b0104480 2024-08-01T04:17:16.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.290+0000 7ff0ae000640 1 -- 172.21.15.3:0/1220397216 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0188f0 con 0x7ff0b0104480 2024-08-01T04:17:16.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.290+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b019f660 con 0x7ff0b0104480 2024-08-01T04:17:16.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.290+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b01a6f40 con 0x7ff0b0104480 2024-08-01T04:17:16.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.294+0000 7ff0ba9d2740 1 Processor -- start 2024-08-01T04:17:16.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.294+0000 7ff0ba9d2740 1 -- start start 2024-08-01T04:17:16.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.294+0000 7ff0ae000640 1 -- 172.21.15.3:0/1220397216 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7ff09c010ac0 con 0x7ff0b0104480 2024-08-01T04:17:16.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.294+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210265c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.294+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x562102672260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5621025777e0 con 0x56210265c3a0 2024-08-01T04:17:16.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5621025c2ac0 con 0x56210266fe70 2024-08-01T04:17:16.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56210255a8c0 con 0x56210265ccc0 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0b5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0b5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55786/0 (socket says 172.21.15.3:55786) 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0b5400640 1 -- 172.21.15.3:0/978971070 learned_addr learned my addr 172.21.15.3:0/978971070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x562102672260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5400640 1 -- 172.21.15.3:0/978971070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 msgr2=0x562102672260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5400640 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x562102672260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5400640 1 -- 172.21.15.3:0/978971070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 msgr2=0x56210265c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.295+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210265c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5400640 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210265c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5400640 1 -- 172.21.15.3:0/978971070 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562102561760 con 0x56210265ccc0 2024-08-01T04:17:16.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210265c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:16.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0b5400640 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266f930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40099b0 tx=0x7ff0a4005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899357a82a74a739 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:16.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.296+0000 7ff0ae000640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff0980a2f20 0x7ff0980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0ae000640 1 -- 172.21.15.3:0/1220397216 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff09c0cfce0 con 0x7ff0b0104480 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0a2600640 1 -- 172.21.15.3:0/978971070 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4037040 con 0x56210265ccc0 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0a2600640 1 -- 172.21.15.3:0/978971070 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4040070 con 0x56210265ccc0 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0a2600640 1 -- 172.21.15.3:0/978971070 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4003850 con 0x56210265ccc0 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff0980a2f20 0x7ff0980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/978971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 msgr2=0x56210266f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.297+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266f930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40099b0 tx=0x7ff0a4005c90 comp rx=0 tx=0).stop 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.298+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/978971070 shutdown_connections 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.298+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x562102672260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.298+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266f930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.298+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/978971070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210265c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.298+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/978971070 >> 172.21.15.3:0/978971070 conn(0x562102658a10 msgr2=0x562102658df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.299+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/978971070 shutdown_connections 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.299+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff0980a2f20 0x7ff0980a53e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009a60 tx=0x7ff0a80095c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:16.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.299+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/978971070 wait complete. 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.299+0000 7ff0ba9d2740 1 -- start start 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.299+0000 7ff0ba9d2740 1 Processor -- start 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210266ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x56210266d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56210257dae0 con 0x56210265c3a0 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5621025bea80 con 0x56210266fe70 2024-08-01T04:17:16.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562102672a20 con 0x56210265ccc0 2024-08-01T04:17:16.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.300+0000 7ff0ba9d2740 10 client.0 _ll_register_callbacks cb 0x56210257f190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:16.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210266ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b5e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x56210266d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b5400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210266ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42080/0 (socket says 172.21.15.3:42080) 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b5400640 1 -- 172.21.15.3:0/24897955 learned_addr learned my addr 172.21.15.3:0/24897955 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b5e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x56210266d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39348/0 (socket says 172.21.15.3:39348) 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b4a00640 1 -- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 msgr2=0x56210266d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x56210266d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b4a00640 1 -- 172.21.15.3:0/24897955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 msgr2=0x56210266ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210266ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.301+0000 7ff0b4a00640 1 -- 172.21.15.3:0/24897955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621023346a0 con 0x56210265ccc0 2024-08-01T04:17:16.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266d360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0cd1b0 tx=0x7ff09c009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd42341615e7063 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:16.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff096a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff096a00640 1 -- 172.21.15.3:0/24897955 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c09c680 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff096a00640 1 -- 172.21.15.3:0/24897955 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff09c09b3d0 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a4040070 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff096a00640 1 -- 172.21.15.3:0/24897955 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff09c09bf80 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff0ba9d2740 10 client.25432 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.302+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56210267e5c0 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff096a00640 1 -- 172.21.15.3:0/24897955 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4040070 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff096a00640 1 -- 172.21.15.3:0/24897955 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff09c097d60 con 0x56210265ccc0 2024-08-01T04:17:16.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0ba9d2740 10 client.25432 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0ba9d2740 10 client.25432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0ba9d2740 20 client.25432 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56210267fa80 0x562102681e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0ba9d2740 4 client.25432 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621025d0030 con 0x56210267fa80 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.303+0000 7ff0b5400640 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56210267fa80 0x562102681e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.304+0000 7ff0b5400640 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56210267fa80 0x562102681e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5514aee62d7be000 server_cookie=2100c666efc07247 in_seq=0 out_seq=0 2024-08-01T04:17:16.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.304+0000 7ff096a00640 10 client.25432 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:16.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.305+0000 7ff096a00640 1 -- 172.21.15.3:0/24897955 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621025d0030 con 0x56210267fa80 2024-08-01T04:17:16.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.305+0000 7ff096a00640 10 client.25432 handle_command_reply: tid=0 2024-08-01T04:17:16.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.305+0000 7ff0ba9d2740 1 client.25432 shutdown 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.305+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 msgr2=0x56210266d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.305+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266d360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0cd1b0 tx=0x7ff09c009700 comp rx=0 tx=0).stop 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 shutdown_connections 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56210267fa80 0x562102681e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff096a00640 0 client.25432 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210266fe70 0x56210266d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x56210266d360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/24897955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x56210266ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 >> 172.21.15.3:0/24897955 conn(0x562102658a10 msgr2=0x56210265e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 shutdown_connections 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/24897955 wait complete. 2024-08-01T04:17:16.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.306+0000 7ff0ba9d2740 20 client.25432 trim_cache size 0 max 16384 2024-08-01T04:17:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.337+0000 7ff0ba9d2740 1 Processor -- start 2024-08-01T04:17:16.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.337+0000 7ff0ba9d2740 1 -- start start 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 0x56210266d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x5621026877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 0x562102687ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01adff0 con 0x56210266fe70 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01ab8c0 con 0x56210265c3a0 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01a9710 con 0x56210265ccc0 2024-08-01T04:17:16.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0b5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 0x562102687ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0b5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 0x56210266d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0b5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 0x562102687ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42088/0 (socket says 172.21.15.3:42088) 2024-08-01T04:17:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0b5e00640 1 -- 172.21.15.3:0/1618599695 learned_addr learned my addr 172.21.15.3:0/1618599695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0b5400640 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 0x56210266d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39350/0 (socket says 172.21.15.3:39350) 2024-08-01T04:17:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.338+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x5621026877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5400640 1 -- 172.21.15.3:0/1618599695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 msgr2=0x5621026877a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5400640 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x5621026877a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5400640 1 -- 172.21.15.3:0/1618599695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 msgr2=0x562102687ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5400640 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 0x562102687ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5400640 1 -- 172.21.15.3:0/1618599695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210267c630 con 0x56210265c3a0 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 0x562102687ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b4a00640 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x5621026877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.339+0000 7ff0b5400640 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 0x56210266d4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40309f0 tx=0x7ff0a4005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2eb7440b4c8d22 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.340+0000 7ff0a1c00640 1 -- 172.21.15.3:0/1618599695 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4037040 con 0x56210265c3a0 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.340+0000 7ff0a1c00640 1 -- 172.21.15.3:0/1618599695 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4040070 con 0x56210265c3a0 2024-08-01T04:17:16.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.340+0000 7ff0a1c00640 1 -- 172.21.15.3:0/1618599695 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4032d10 con 0x56210265c3a0 2024-08-01T04:17:16.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/1618599695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 msgr2=0x56210266d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 0x56210266d4f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40309f0 tx=0x7ff0a4005ab0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/1618599695 shutdown_connections 2024-08-01T04:17:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210266fe70 0x562102687ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210265ccc0 0x5621026877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/1618599695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265c3a0 0x56210266d4f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/1618599695 >> 172.21.15.3:0/1618599695 conn(0x562102658a10 msgr2=0x56210266cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/1618599695 shutdown_connections 2024-08-01T04:17:16.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.341+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/1618599695 wait complete. 2024-08-01T04:17:16.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.342+0000 7ff0ba9d2740 1 -- start start 2024-08-01T04:17:16.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.342+0000 7ff0ba9d2740 1 Processor -- start 2024-08-01T04:17:16.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.342+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x562102674100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265ccc0 0x562102674640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0ba9d2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 0x5621026e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01a9d80 con 0x56210265c3a0 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01ab1f0 con 0x56210265ccc0 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0ba9d2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b01aa450 con 0x56210266fe70 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0ba9d2740 10 client.0 _ll_register_callbacks cb 0x7ff09c0d3480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0b5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 0x5621026e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0b5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 0x5621026e9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55818/0 (socket says 172.21.15.3:55818) 2024-08-01T04:17:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.343+0000 7ff0b5e00640 1 -- 172.21.15.3:0/3819174139 learned_addr learned my addr 172.21.15.3:0/3819174139 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5e00640 1 -- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265ccc0 msgr2=0x562102674640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265ccc0 0x562102674640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5400640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x562102674100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5e00640 1 -- 172.21.15.3:0/3819174139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 msgr2=0x562102674100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:16.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x562102674100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5e00640 1 -- 172.21.15.3:0/3819174139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562102667c40 con 0x56210266fe70 2024-08-01T04:17:16.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5400640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x562102674100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:16.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff0b5e00640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 0x5621026e9770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff0a801bf00 tx=0x7ff0a8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd02d67c1696848f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:16.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.344+0000 7ff096a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:16.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.345+0000 7ff096a00640 1 -- 172.21.15.3:0/3819174139 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a802c070 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.345+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a4040070 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.345+0000 7ff096a00640 1 -- 172.21.15.3:0/3819174139 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff0a801ac30 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.345+0000 7ff096a00640 1 -- 172.21.15.3:0/3819174139 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0a8038050 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.345+0000 7ff0ba9d2740 10 client.25438 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.345+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5621026eae70 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff096a00640 1 -- 172.21.15.3:0/3819174139 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a4040070 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff096a00640 1 -- 172.21.15.3:0/3819174139 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff0a800b040 con 0x56210266fe70 2024-08-01T04:17:16.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff0ba9d2740 10 client.25438 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:16.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff0ba9d2740 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:16.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff0ba9d2740 20 client.25438 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:16.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5621026eb570 0x5621026ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:16.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff0ba9d2740 4 client.25438 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:17:16.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.346+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5621025ca540 con 0x5621026eb570 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.347+0000 7ff0b5400640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5621026eb570 0x5621026ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.347+0000 7ff0b5400640 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5621026eb570 0x5621026ed9f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec03808503caf9ac server_cookie=7be2283289f33d6a in_seq=0 out_seq=0 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.347+0000 7ff096a00640 10 client.25438 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.348+0000 7ff096a00640 1 -- 172.21.15.3:0/3819174139 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5621025ca540 con 0x5621026eb570 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.348+0000 7ff096a00640 10 client.25438 handle_command_reply: tid=0 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.348+0000 7ff0ba9d2740 1 client.25438 shutdown 2024-08-01T04:17:16.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 msgr2=0x5621026e9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 0x5621026e9770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff0a801bf00 tx=0x7ff0a8004a40 comp rx=0 tx=0).stop 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 shutdown_connections 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5621026eb570 0x5621026ed9f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff096a00640 0 client.25438 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56210266fe70 0x5621026e9770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56210265ccc0 0x562102674640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 --2- 172.21.15.3:0/3819174139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56210265c3a0 0x562102674100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 >> 172.21.15.3:0/3819174139 conn(0x562102658a10 msgr2=0x56210266eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:16.356 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:16.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.349+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 shutdown_connections 2024-08-01T04:17:16.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.350+0000 7ff0ba9d2740 1 -- 172.21.15.3:0/3819174139 wait complete. 2024-08-01T04:17:16.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.350+0000 7ff0ba9d2740 20 client.25438 trim_cache size 0 max 16384 2024-08-01T04:17:16.357 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:17:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "43809979-acdb-492f-9c01-14d7219597f4", 2024-08-01T04:17:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:17:16.357 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:16.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff0980a2f20 msgr2=0x7ff0980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff0980a2f20 0x7ff0980a53e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009a60 tx=0x7ff0a80095c0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 msgr2=0x7ff0b019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 0x7ff0b019a900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e9f0 tx=0x7ff09c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 shutdown_connections 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0b0109b70 0x7ff0b019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff0980a2f20 0x7ff0980a53e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff0b0104480 0x7ff0b019a900 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 --2- 172.21.15.3:0/1220397216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0b0103ae0 0x7ff0b019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.351+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 >> 172.21.15.3:0/1220397216 conn(0x7ff0b00ff780 msgr2=0x7ff0b01010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.352+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 shutdown_connections 2024-08-01T04:17:16.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:16.352+0000 7ff0b7e00640 1 -- 172.21.15.3:0/1220397216 wait complete. 2024-08-01T04:17:17.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:17 smithi163 ceph-mon[37383]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-01T04:17:17.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:17 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:17:17.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:17 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:17.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:17 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:17:17.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:17 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:17.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:17 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:17:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:17 smithi003 ceph-mon[32262]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-01T04:17:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:17 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:17:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:17 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:17 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:17:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:17 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:17.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:17 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:17:17.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 -- 172.21.15.3:0/4046251638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec10bcf0 msgr2=0x7f3bec10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 --2- 172.21.15.3:0/4046251638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec10bcf0 0x7f3bec10e190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009ca0 tx=0x7f3be402a950 comp rx=0 tx=0).stop 2024-08-01T04:17:17.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 -- 172.21.15.3:0/4046251638 shutdown_connections 2024-08-01T04:17:17.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 --2- 172.21.15.3:0/4046251638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec10bcf0 0x7f3bec10e190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 --2- 172.21.15.3:0/4046251638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bec1031e0 0x7f3bec10b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 --2- 172.21.15.3:0/4046251638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bec1028c0 0x7f3bec102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 -- 172.21.15.3:0/4046251638 >> 172.21.15.3:0/4046251638 conn(0x7f3bec0fc340 msgr2=0x7f3bec0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:17.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 -- 172.21.15.3:0/4046251638 shutdown_connections 2024-08-01T04:17:17.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.382+0000 7f3bf2800640 1 -- 172.21.15.3:0/4046251638 wait complete. 2024-08-01T04:17:17.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.384+0000 7f3bf2800640 1 Processor -- start 2024-08-01T04:17:17.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.384+0000 7f3bf2800640 1 -- start start 2024-08-01T04:17:17.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf2800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 0x7f3bec100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf2800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bec1031e0 0x7f3bec100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf2800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bec10bcf0 0x7f3bec100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf2800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec111b00 con 0x7f3bec1028c0 2024-08-01T04:17:17.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf2800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec111980 con 0x7f3bec10bcf0 2024-08-01T04:17:17.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf2800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec111c80 con 0x7f3bec1031e0 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 0x7f3bec100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 0x7f3bec100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42128/0 (socket says 172.21.15.3:42128) 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 -- 172.21.15.3:0/743791679 learned_addr learned my addr 172.21.15.3:0/743791679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bec1031e0 0x7f3bec100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bec10bcf0 0x7f3bec100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 -- 172.21.15.3:0/743791679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bec1031e0 msgr2=0x7f3bec100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bec1031e0 0x7f3bec100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 -- 172.21.15.3:0/743791679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bec10bcf0 msgr2=0x7f3bec100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bec10bcf0 0x7f3bec100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.385+0000 7f3bf1400640 1 -- 172.21.15.3:0/743791679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec1a6440 con 0x7f3bec1028c0 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.386+0000 7f3bf1400640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 0x7f3bec100520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9f0 tx=0x7f3be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.386+0000 7f3bea000640 1 -- 172.21.15.3:0/743791679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be000cd80 con 0x7f3bec1028c0 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.386+0000 7f3bea000640 1 -- 172.21.15.3:0/743791679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be0010910 con 0x7f3bec1028c0 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.386+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec1a6730 con 0x7f3bec1028c0 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.386+0000 7f3bea000640 1 -- 172.21.15.3:0/743791679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0018950 con 0x7f3bec1028c0 2024-08-01T04:17:17.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.386+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec1a6c70 con 0x7f3bec1028c0 2024-08-01T04:17:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.390+0000 7f3bea000640 1 -- 172.21.15.3:0/743791679 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f3be0010ac0 con 0x7f3bec1028c0 2024-08-01T04:17:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.390+0000 7f3bf67ca740 1 Processor -- start 2024-08-01T04:17:17.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.390+0000 7f3bf67ca740 1 -- start start 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f19030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f19570 0x557b51f199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f26700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557b51e7eb90 con 0x557b51f19570 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557b51e161d0 con 0x557b51f1aae0 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557b51e7ab50 con 0x557b51f24310 2024-08-01T04:17:17.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f19570 0x557b51f199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f19570 0x557b51f199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42140/0 (socket says 172.21.15.3:42140) 2024-08-01T04:17:17.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf0a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f26700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf1400640 1 -- 172.21.15.3:0/181892307 learned_addr learned my addr 172.21.15.3:0/181892307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:17.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.391+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f19030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf0a00640 1 -- 172.21.15.3:0/181892307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 msgr2=0x557b51f19030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f19030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf0a00640 1 -- 172.21.15.3:0/181892307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f19570 msgr2=0x557b51f199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f19570 0x557b51f199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf0a00640 1 -- 172.21.15.3:0/181892307 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b51bf0b90 con 0x557b51f24310 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f26700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009be0 tx=0x7f3bd8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79f85bcb58c570ae server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bea000640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3bd40a2ea0 0x7f3bd40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bea000640 1 -- 172.21.15.3:0/743791679 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3be00cfb40 con 0x7f3bec1028c0 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bde600640 1 -- 172.21.15.3:0/181892307 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8004050 con 0x557b51f24310 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.392+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3bd40a2ea0 0x7f3bd40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.393+0000 7f3bde600640 1 -- 172.21.15.3:0/181892307 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8005500 con 0x557b51f24310 2024-08-01T04:17:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.393+0000 7f3bde600640 1 -- 172.21.15.3:0/181892307 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8004d10 con 0x557b51f24310 2024-08-01T04:17:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 -- 172.21.15.3:0/181892307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 msgr2=0x557b51f26700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3bd40a2ea0 0x7f3bd40a5360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be4005d80 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f26700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009be0 tx=0x7f3bd8011810 comp rx=0 tx=0).stop 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 -- 172.21.15.3:0/181892307 shutdown_connections 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f26700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f19570 0x557b51f199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/181892307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f19030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 -- 172.21.15.3:0/181892307 >> 172.21.15.3:0/181892307 conn(0x557b51ee2330 msgr2=0x557b51f15c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 -- 172.21.15.3:0/181892307 shutdown_connections 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.394+0000 7f3bf67ca740 1 -- 172.21.15.3:0/181892307 wait complete. 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 -- start start 2024-08-01T04:17:17.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 Processor -- start 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f21270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f21cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557b51f27010 con 0x557b51f24310 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557b51e39920 con 0x557b51f1aae0 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557b51f27190 con 0x557b51f19570 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.395+0000 7f3bf67ca740 10 client.0 _ll_register_callbacks cb 0x557b51b8e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f21270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f21270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55854/0 (socket says 172.21.15.3:55854) 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf0a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 -- 172.21.15.3:0/982487270 learned_addr learned my addr 172.21.15.3:0/982487270 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f21cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 -- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 msgr2=0x557b51f217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 -- 172.21.15.3:0/982487270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 msgr2=0x557b51f21cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f21cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.396+0000 7f3bf1400640 1 -- 172.21.15.3:0/982487270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b51f305f0 con 0x557b51f19570 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bf1400640 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f21270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3be00987b0 tx=0x7f3be0005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21894298eec1ef47 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bd2a00640 1 -- 172.21.15.3:0/982487270 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be009c4e0 con 0x557b51f19570 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bd2a00640 1 -- 172.21.15.3:0/982487270 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3be0010430 con 0x557b51f19570 2024-08-01T04:17:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd8005500 con 0x557b51f19570 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bd2a00640 1 -- 172.21.15.3:0/982487270 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3be009bde0 con 0x557b51f19570 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bf67ca740 10 client.25450 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.397+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b51f32fd0 con 0x557b51f19570 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bd2a00640 1 -- 172.21.15.3:0/982487270 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd8005500 con 0x557b51f19570 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bd2a00640 1 -- 172.21.15.3:0/982487270 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3be0097d10 con 0x557b51f19570 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bf67ca740 10 client.25450 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bf67ca740 10 client.25450 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bf67ca740 20 client.25450 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f34490 0x557b51f36900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bf67ca740 4 client.25450 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.398+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b51e53320 con 0x557b51f34490 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.399+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f34490 0x557b51f36900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.399+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f34490 0x557b51f36900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11be01225b814f53 server_cookie=b13613af65b74232 in_seq=0 out_seq=0 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.399+0000 7f3bd2a00640 10 client.25450 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bd2a00640 1 -- 172.21.15.3:0/982487270 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b51e53320 con 0x557b51f34490 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bd2a00640 10 client.25450 handle_command_reply: tid=0 2024-08-01T04:17:17.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bf67ca740 1 client.25450 shutdown 2024-08-01T04:17:17.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:17 smithi160 ceph-mon[37224]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-01T04:17:17.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:17 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:17:17.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:17 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:17.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:17 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:17:17.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:17 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:17.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:17 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 msgr2=0x557b51f21270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f21270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3be00987b0 tx=0x7f3be0005760 comp rx=0 tx=0).stop 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 shutdown_connections 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.400+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f34490 0x557b51f36900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f21cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bd2a00640 0 client.25450 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/982487270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f21270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 >> 172.21.15.3:0/982487270 conn(0x557b51ee2330 msgr2=0x557b51f1b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 shutdown_connections 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 1 -- 172.21.15.3:0/982487270 wait complete. 2024-08-01T04:17:17.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.401+0000 7f3bf67ca740 20 client.25450 trim_cache size 0 max 16384 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 Processor -- start 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 -- start start 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f3c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f3c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b2030 con 0x557b51f24310 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1af900 con 0x557b51f1aae0 2024-08-01T04:17:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.425+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1ad750 con 0x557b51f19570 2024-08-01T04:17:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55868/0 (socket says 172.21.15.3:55868) 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 -- 172.21.15.3:0/956195615 learned_addr learned my addr 172.21.15.3:0/956195615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f3c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f3c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 -- 172.21.15.3:0/956195615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 msgr2=0x557b51f3c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f3c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 -- 172.21.15.3:0/956195615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 msgr2=0x557b51f3c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f3c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 -- 172.21.15.3:0/956195615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b51f20220 con 0x557b51f19570 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.426+0000 7f3bf1400640 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f231e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3be009a090 tx=0x7f3be009b100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549b34b8e0d945dc server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bddc00640 1 -- 172.21.15.3:0/956195615 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00a9070 con 0x557b51f19570 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bddc00640 1 -- 172.21.15.3:0/956195615 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be009c4e0 con 0x557b51f19570 2024-08-01T04:17:17.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bddc00640 1 -- 172.21.15.3:0/956195615 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0018d30 con 0x557b51f19570 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bf67ca740 1 -- 172.21.15.3:0/956195615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 msgr2=0x557b51f231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f231e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3be009a090 tx=0x7f3be009b100 comp rx=0 tx=0).stop 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bf67ca740 1 -- 172.21.15.3:0/956195615 shutdown_connections 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.427+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f24310 0x557b51f3c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.428+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f1aae0 0x557b51f3c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.428+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/956195615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f19570 0x557b51f231e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.428+0000 7f3bf67ca740 1 -- 172.21.15.3:0/956195615 >> 172.21.15.3:0/956195615 conn(0x557b51ee2330 msgr2=0x557b51f22ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.428+0000 7f3bf67ca740 1 -- 172.21.15.3:0/956195615 shutdown_connections 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.428+0000 7f3bf67ca740 1 -- 172.21.15.3:0/956195615 wait complete. 2024-08-01T04:17:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.428+0000 7f3bf67ca740 1 -- start start 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 Processor -- start 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f19570 0x557b51f2cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 0x557b51f27e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1addc0 con 0x557b51f1aae0 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1af230 con 0x557b51f19570 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1ae490 con 0x557b51f24310 2024-08-01T04:17:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf67ca740 10 client.0 _ll_register_callbacks cb 0x7f3be000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 0x557b51f27e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf0a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 0x557b51f27e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42160/0 (socket says 172.21.15.3:42160) 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf0a00640 1 -- 172.21.15.3:0/4290559862 learned_addr learned my addr 172.21.15.3:0/4290559862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.429+0000 7f3bf1400640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f19570 0x557b51f2cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf1e00640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf0a00640 1 -- 172.21.15.3:0/4290559862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 msgr2=0x557b51f28350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f28350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf0a00640 1 -- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f19570 msgr2=0x557b51f2cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f19570 0x557b51f2cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf0a00640 1 -- 172.21.15.3:0/4290559862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b51f28b30 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bf0a00640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 0x557b51f27e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009bb0 tx=0x7f3bd8004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a936172a97802c8b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.430+0000 7f3bd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bd2a00640 1 -- 172.21.15.3:0/4290559862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8032070 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bd2a00640 1 -- 172.21.15.3:0/4290559862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3bd8004250 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be009c4e0 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bd2a00640 1 -- 172.21.15.3:0/4290559862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bd8004450 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bf67ca740 10 client.15333 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b51f9f120 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.431+0000 7f3bd2a00640 1 -- 172.21.15.3:0/4290559862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be009c4e0 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bd2a00640 1 -- 172.21.15.3:0/4290559862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3bd8035020 con 0x557b51f1aae0 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf67ca740 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf67ca740 10 client.15333 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:17:17.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf67ca740 20 client.15333 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f9f7c0 0x557b51fa1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf67ca740 4 client.15333 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.4, "await_for": 45.0}] 2024-08-01T04:17:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.4, "await_for": 45.0}) -- 0x557b51e8c2c0 con 0x557b51f9f7c0 2024-08-01T04:17:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.432+0000 7f3bf1400640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f9f7c0 0x557b51fa1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.433+0000 7f3bf1400640 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f9f7c0 0x557b51fa1be0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8710eb7fb368a771 server_cookie=b74aa686aece013a in_seq=0 out_seq=0 2024-08-01T04:17:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.433+0000 7f3bd2a00640 10 client.15333 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.443+0000 7f3bd2a00640 1 -- 172.21.15.3:0/4290559862 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557b51e8c2c0 con 0x557b51f9f7c0 2024-08-01T04:17:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.443+0000 7f3bd2a00640 10 client.15333 handle_command_reply: tid=0 2024-08-01T04:17:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.443+0000 7f3bf67ca740 1 client.15333 shutdown 2024-08-01T04:17:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 msgr2=0x557b51f27e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 0x557b51f27e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009bb0 tx=0x7f3bd8004860 comp rx=0 tx=0).stop 2024-08-01T04:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 shutdown_connections 2024-08-01T04:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557b51f9f7c0 0x557b51fa1be0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557b51f24310 0x557b51f28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557b51f1aae0 0x557b51f27e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 --2- 172.21.15.3:0/4290559862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557b51f19570 0x557b51f2cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 >> 172.21.15.3:0/4290559862 conn(0x557b51ee2330 msgr2=0x557b51f232f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bd2a00640 0 client.15333 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 shutdown_connections 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 1 -- 172.21.15.3:0/4290559862 wait complete. 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.444+0000 7f3bf67ca740 20 client.15333 trim_cache size 0 max 16384 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-08-01T04:17:17.447 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 2, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "5c87d78e": { 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 2, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.4, 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:17:17.448 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:17.449 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3bd40a2ea0 msgr2=0x7f3bd40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3bd40a2ea0 0x7f3bd40a5360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3be4006010 tx=0x7f3be4005d80 comp rx=0 tx=0).stop 2024-08-01T04:17:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 msgr2=0x7f3bec100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 0x7f3bec100520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9f0 tx=0x7f3be000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 shutdown_connections 2024-08-01T04:17:17.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bec10bcf0 0x7f3bec100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3bd40a2ea0 0x7f3bd40a5360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bec1031e0 0x7f3bec100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 --2- 172.21.15.3:0/743791679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bec1028c0 0x7f3bec100520 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.445+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 >> 172.21.15.3:0/743791679 conn(0x7f3bec0fc340 msgr2=0x7f3bec0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.446+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 shutdown_connections 2024-08-01T04:17:17.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:17.446+0000 7f3bf2800640 1 -- 172.21.15.3:0/743791679 wait complete. 2024-08-01T04:17:17.740 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:17:18.587 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5c87d78e', quiesce took 6.0 seconds. Will release after: 18.4 2024-08-01T04:17:18.712 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:19.123 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:19 smithi003 ceph-mon[32262]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-01T04:17:19.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:19 smithi163 ceph-mon[37383]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-01T04:17:19.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:19 smithi160 ceph-mon[37224]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 -- 172.21.15.3:0/1000295005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80103ac0 msgr2=0x7f0c80103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 --2- 172.21.15.3:0/1000295005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80103ac0 0x7f0c80103f20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009930 tx=0x7f0c6c02a440 comp rx=0 tx=0).stop 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 -- 172.21.15.3:0/1000295005 shutdown_connections 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 --2- 172.21.15.3:0/1000295005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80104460 0x7f0c80110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 --2- 172.21.15.3:0/1000295005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80103ac0 0x7f0c80103f20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 --2- 172.21.15.3:0/1000295005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c80109ac0 0x7f0c80109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 -- 172.21.15.3:0/1000295005 >> 172.21.15.3:0/1000295005 conn(0x7f0c800ff7c0 msgr2=0x7f0c80101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 -- 172.21.15.3:0/1000295005 shutdown_connections 2024-08-01T04:17:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.419+0000 7f0c85400640 1 -- 172.21.15.3:0/1000295005 wait complete. 2024-08-01T04:17:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.420+0000 7f0c85400640 1 Processor -- start 2024-08-01T04:17:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.420+0000 7f0c85400640 1 -- start start 2024-08-01T04:17:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.420+0000 7f0c85400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c80103ac0 0x7f0c8019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.420+0000 7f0c85400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 0x7f0c8019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c85400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80109ac0 0x7f0c8019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c85400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113f10 con 0x7f0c80104460 2024-08-01T04:17:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c85400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80113d90 con 0x7f0c80109ac0 2024-08-01T04:17:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c85400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80114090 con 0x7f0c80103ac0 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 0x7f0c8019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 0x7f0c8019a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42188/0 (socket says 172.21.15.3:42188) 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 -- 172.21.15.3:0/430689753 learned_addr learned my addr 172.21.15.3:0/430689753 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7e800640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c80103ac0 0x7f0c8019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7f200640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80109ac0 0x7f0c8019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 -- 172.21.15.3:0/430689753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c80103ac0 msgr2=0x7f0c8019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c80103ac0 0x7f0c8019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 -- 172.21.15.3:0/430689753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80109ac0 msgr2=0x7f0c8019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80109ac0 0x7f0c8019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7de00640 1 -- 172.21.15.3:0/430689753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8019f360 con 0x7f0c80104460 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.421+0000 7f0c7f200640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80109ac0 0x7f0c8019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.422+0000 7f0c7de00640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 0x7f0c8019a8f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c02a960 tx=0x7f0c6c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.422+0000 7f0c77400640 1 -- 172.21.15.3:0/430689753 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c03f070 con 0x7f0c80104460 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.422+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8019f5f0 con 0x7f0c80104460 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.422+0000 7f0c77400640 1 -- 172.21.15.3:0/430689753 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c6c031c90 con 0x7f0c80104460 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.422+0000 7f0c77400640 1 -- 172.21.15.3:0/430689753 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c035030 con 0x7f0c80104460 2024-08-01T04:17:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.422+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801a6f30 con 0x7f0c80104460 2024-08-01T04:17:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.427+0000 7f0c87de4740 1 Processor -- start 2024-08-01T04:17:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.427+0000 7f0c77400640 1 -- 172.21.15.3:0/430689753 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f0c6c042070 con 0x7f0c80104460 2024-08-01T04:17:19.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 -- start start 2024-08-01T04:17:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae15f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f94b0 0x560ae1603830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae1606160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560ae155ebf0 con 0x560ae15f94b0 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560ae14f6bd0 con 0x560ae15f8b90 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c87de4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560ae155abb0 con 0x560ae1603d70 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c7f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae15f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c7f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae15f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39442/0 (socket says 172.21.15.3:39442) 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.428+0000 7f0c7f200640 1 -- 172.21.15.3:0/2445760574 learned_addr learned my addr 172.21.15.3:0/2445760574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7de00640 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae1606160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7f200640 1 -- 172.21.15.3:0/2445760574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 msgr2=0x560ae1606160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7f200640 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae1606160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7f200640 1 -- 172.21.15.3:0/2445760574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f94b0 msgr2=0x560ae1603830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7f200640 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f94b0 0x560ae1603830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7f200640 1 -- 172.21.15.3:0/2445760574 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae14fd8a0 con 0x560ae15f8b90 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c7f200640 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae15f8f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c780107e0 tx=0x7f0c78010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46fd87a641e092f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c6b400640 1 -- 172.21.15.3:0/2445760574 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78003b70 con 0x560ae15f8b90 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c6b400640 1 -- 172.21.15.3:0/2445760574 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7800ee10 con 0x560ae15f8b90 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.429+0000 7f0c6b400640 1 -- 172.21.15.3:0/2445760574 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780047f0 con 0x560ae15f8b90 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.430+0000 7f0c77400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0c640a2ea0 0x7f0c640a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.430+0000 7f0c77400640 1 -- 172.21.15.3:0/430689753 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f0c6c0efae0 con 0x7f0c80104460 2024-08-01T04:17:19.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.430+0000 7f0c7e800640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0c640a2ea0 0x7f0c640a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.430+0000 7f0c87de4740 1 -- 172.21.15.3:0/2445760574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 msgr2=0x560ae15f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.430+0000 7f0c87de4740 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae15f8f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c780107e0 tx=0x7f0c78010ca0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 -- 172.21.15.3:0/2445760574 shutdown_connections 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae1606160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f94b0 0x560ae1603830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 --2- 172.21.15.3:0/2445760574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae15f8f70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 -- 172.21.15.3:0/2445760574 >> 172.21.15.3:0/2445760574 conn(0x560ae15f5120 msgr2=0x560ae15f5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 -- 172.21.15.3:0/2445760574 shutdown_connections 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 -- 172.21.15.3:0/2445760574 wait complete. 2024-08-01T04:17:19.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c7e800640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0c640a2ea0 0x7f0c640a5360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x560ae15f5f80 tx=0x7f0c7000b750 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:19.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.431+0000 7f0c87de4740 1 -- start start 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 Processor -- start 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f8b90 0x560ae1600cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae1601200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae1601740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560ae1606940 con 0x560ae1603d70 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560ae1519d40 con 0x560ae15f94b0 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560ae1606ac0 con 0x560ae15f8b90 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c87de4740 10 client.0 _ll_register_callbacks cb 0x560ae14e2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.432+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae1601200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae1601200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39444/0 (socket says 172.21.15.3:39444) 2024-08-01T04:17:19.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae1601740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 -- 172.21.15.3:0/4237256216 learned_addr learned my addr 172.21.15.3:0/4237256216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7e800640 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f8b90 0x560ae1600cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 -- 172.21.15.3:0/4237256216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f8b90 msgr2=0x560ae1600cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f8b90 0x560ae1600cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 -- 172.21.15.3:0/4237256216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 msgr2=0x560ae1601740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae1601740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 -- 172.21.15.3:0/4237256216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae12d03c0 con 0x560ae15f94b0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c7de00640 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae1601200 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0bbdc0 tx=0x7f0c6c0b80d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dedc67f3b4eb5d9f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.433+0000 7f0c5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c5fe00640 1 -- 172.21.15.3:0/4237256216 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c049070 con 0x560ae15f94b0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c5fe00640 1 -- 172.21.15.3:0/4237256216 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f0c6c0c4560 con 0x560ae15f94b0 2024-08-01T04:17:19.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7800ee10 con 0x560ae15f94b0 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c5fe00640 1 -- 172.21.15.3:0/4237256216 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c6c0bb790 con 0x560ae15f94b0 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c87de4740 10 client.25478 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ae1606940 con 0x560ae15f94b0 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.434+0000 7f0c5fe00640 1 -- 172.21.15.3:0/4237256216 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7800ee10 con 0x560ae15f94b0 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c5fe00640 1 -- 172.21.15.3:0/4237256216 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0c6c0d2030 con 0x560ae15f94b0 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c87de4740 10 client.25478 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c87de4740 10 client.25478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c87de4740 20 client.25478 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c87de4740 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae1613a50 0x560ae1615ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c87de4740 4 client.25478 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:19.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ae156c160 con 0x560ae1613a50 2024-08-01T04:17:19.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.435+0000 7f0c7e800640 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae1613a50 0x560ae1615ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.436+0000 7f0c7e800640 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae1613a50 0x560ae1615ef0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c9b70b5afc4066 server_cookie=9f37c966123219fe in_seq=0 out_seq=0 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.436+0000 7f0c5fe00640 10 client.25478 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c5fe00640 1 -- 172.21.15.3:0/4237256216 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ae156c160 con 0x560ae1613a50 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c5fe00640 10 client.25478 handle_command_reply: tid=0 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 client.25478 shutdown 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 msgr2=0x560ae1601200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae1601200 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0bbdc0 tx=0x7f0c6c0b80d0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 shutdown_connections 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae1613a50 0x560ae1615ef0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae1601740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c5fe00640 0 client.25478 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae1601200 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 --2- 172.21.15.3:0/4237256216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f8b90 0x560ae1600cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.437+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 >> 172.21.15.3:0/4237256216 conn(0x560ae15f5120 msgr2=0x560ae15fae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.438+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 shutdown_connections 2024-08-01T04:17:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.438+0000 7f0c87de4740 1 -- 172.21.15.3:0/4237256216 wait complete. 2024-08-01T04:17:19.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.438+0000 7f0c87de4740 20 client.25478 trim_cache size 0 max 16384 2024-08-01T04:17:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 Processor -- start 2024-08-01T04:17:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 -- start start 2024-08-01T04:17:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 0x560ae15fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae15f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae15f4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801b2210 con 0x560ae15f8b90 2024-08-01T04:17:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801afae0 con 0x560ae15f94b0 2024-08-01T04:17:19.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.475+0000 7f0c87de4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ad930 con 0x560ae1603d70 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 0x560ae15fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 0x560ae15fb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42216/0 (socket says 172.21.15.3:42216) 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 -- 172.21.15.3:0/100440688 learned_addr learned my addr 172.21.15.3:0/100440688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7f200640 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae15f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 -- 172.21.15.3:0/100440688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 msgr2=0x560ae15f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae15f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 -- 172.21.15.3:0/100440688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 msgr2=0x560ae15f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae15f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.476+0000 7f0c7e800640 1 -- 172.21.15.3:0/100440688 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae15ffc70 con 0x560ae15f8b90 2024-08-01T04:17:19.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.477+0000 7f0c7e800640 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 0x560ae15fb640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x560ae16167a0 tx=0x7f0c70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dc8c9b05b656f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:19.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.477+0000 7f0c6aa00640 1 -- 172.21.15.3:0/100440688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7002d070 con 0x560ae15f8b90 2024-08-01T04:17:19.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.477+0000 7f0c6aa00640 1 -- 172.21.15.3:0/100440688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c700277b0 con 0x560ae15f8b90 2024-08-01T04:17:19.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.477+0000 7f0c6aa00640 1 -- 172.21.15.3:0/100440688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7001e930 con 0x560ae15f8b90 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 -- 172.21.15.3:0/100440688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 msgr2=0x560ae15fb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 0x560ae15fb640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x560ae16167a0 tx=0x7f0c70013000 comp rx=0 tx=0).stop 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 -- 172.21.15.3:0/100440688 shutdown_connections 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae1603d70 0x560ae15f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f94b0 0x560ae15f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 --2- 172.21.15.3:0/100440688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae15f8b90 0x560ae15fb640 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 -- 172.21.15.3:0/100440688 >> 172.21.15.3:0/100440688 conn(0x560ae15f5120 msgr2=0x560ae15f3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 -- 172.21.15.3:0/100440688 shutdown_connections 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.478+0000 7f0c87de4740 1 -- 172.21.15.3:0/100440688 wait complete. 2024-08-01T04:17:19.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.479+0000 7f0c87de4740 1 -- start start 2024-08-01T04:17:19.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.479+0000 7f0c87de4740 1 Processor -- start 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae16081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f94b0 0x560ae1608720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae167d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801adfa0 con 0x560ae1603d70 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801af410 con 0x560ae15f8b90 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801ae670 con 0x560ae15f94b0 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c87de4740 10 client.0 _ll_register_callbacks cb 0x7f0c6c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae167d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae167d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42230/0 (socket says 172.21.15.3:42230) 2024-08-01T04:17:19.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 -- 172.21.15.3:0/723208896 learned_addr learned my addr 172.21.15.3:0/723208896 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7e800640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae16081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7de00640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f94b0 0x560ae1608720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 -- 172.21.15.3:0/723208896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f94b0 msgr2=0x560ae1608720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f94b0 0x560ae1608720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 -- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 msgr2=0x560ae16081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae16081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7f200640 1 -- 172.21.15.3:0/723208896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae167dd10 con 0x560ae1603d70 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.480+0000 7f0c7e800640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae16081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.481+0000 7f0c7f200640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae167d7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c78010700 tx=0x7f0c78005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e185524a69a960da server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.481+0000 7f0c5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.481+0000 7f0c5fe00640 1 -- 172.21.15.3:0/723208896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7802c060 con 0x560ae1603d70 2024-08-01T04:17:19.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.481+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c700277b0 con 0x560ae1603d70 2024-08-01T04:17:19.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.481+0000 7f0c5fe00640 1 -- 172.21.15.3:0/723208896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f0c780056b0 con 0x560ae1603d70 2024-08-01T04:17:19.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c5fe00640 1 -- 172.21.15.3:0/723208896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c780058b0 con 0x560ae1603d70 2024-08-01T04:17:19.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 10 client.15348 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ae167f050 con 0x560ae1603d70 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c5fe00640 1 -- 172.21.15.3:0/723208896 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c700277b0 con 0x560ae1603d70 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c5fe00640 1 -- 172.21.15.3:0/723208896 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0c78026070 con 0x560ae1603d70 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 10 client.15348 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 20 client.15348 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae167f570 0x560ae1681990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 4 client.15348 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.482+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ae1566670 con 0x560ae167f570 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.483+0000 7f0c7e800640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae167f570 0x560ae1681990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.483+0000 7f0c7e800640 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae167f570 0x560ae1681990 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3e33cb347ec7d server_cookie=a4b142a8505bf2af in_seq=0 out_seq=0 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.483+0000 7f0c5fe00640 10 client.15348 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:19.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c5fe00640 1 -- 172.21.15.3:0/723208896 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560ae1566670 con 0x560ae167f570 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c5fe00640 10 client.15348 handle_command_reply: tid=0 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 client.15348 shutdown 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 msgr2=0x560ae167d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae167d7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c78010700 tx=0x7f0c78005540 comp rx=0 tx=0).stop 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 shutdown_connections 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560ae167f570 0x560ae1681990 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560ae1603d70 0x560ae167d7d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560ae15f94b0 0x560ae1608720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 --2- 172.21.15.3:0/723208896 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560ae15f8b90 0x560ae16081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 >> 172.21.15.3:0/723208896 conn(0x560ae15f5120 msgr2=0x560ae16014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c5fe00640 0 client.15348 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 shutdown_connections 2024-08-01T04:17:19.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 1 -- 172.21.15.3:0/723208896 wait complete. 2024-08-01T04:17:19.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.484+0000 7f0c87de4740 20 client.15348 trim_cache size 0 max 16384 2024-08-01T04:17:19.489 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:17:19.489 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:17:19.489 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0c640a2ea0 msgr2=0x7f0c640a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0c640a2ea0 0x7f0c640a5360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x560ae15f5f80 tx=0x7f0c7000b750 comp rx=0 tx=0).stop 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 msgr2=0x7f0c8019a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 0x7f0c8019a8f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c02a960 tx=0x7f0c6c003b90 comp rx=0 tx=0).stop 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 shutdown_connections 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0c80109ac0 0x7f0c8019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c80104460 0x7f0c8019a8f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0c640a2ea0 0x7f0c640a5360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 --2- 172.21.15.3:0/430689753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c80103ac0 0x7f0c8019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:19.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.485+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 >> 172.21.15.3:0/430689753 conn(0x7f0c800ff7c0 msgr2=0x7f0c800fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:19.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.486+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 shutdown_connections 2024-08-01T04:17:19.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:19.486+0000 7f0c85400640 1 -- 172.21.15.3:0/430689753 wait complete. 2024-08-01T04:17:20.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:17:20.365 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:17:20.829 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:21 smithi003 ceph-mon[32262]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:17:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:21 smithi160 ceph-mon[37224]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:17:21.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:21 smithi163 ceph-mon[37383]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:17:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 -- 172.21.15.3:0/3670453226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd74103900 msgr2=0x7fbd74103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 --2- 172.21.15.3:0/3670453226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd74103900 0x7fbd74103d60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009930 tx=0x7fbd6002a420 comp rx=0 tx=0).stop 2024-08-01T04:17:21.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 -- 172.21.15.3:0/3670453226 shutdown_connections 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 --2- 172.21.15.3:0/3670453226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd741042a0 0x7fbd741107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 --2- 172.21.15.3:0/3670453226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd74103900 0x7fbd74103d60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 --2- 172.21.15.3:0/3670453226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74109900 0x7fbd74109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.666+0000 7fbd79000640 1 -- 172.21.15.3:0/3670453226 >> 172.21.15.3:0/3670453226 conn(0x7fbd740ff640 msgr2=0x7fbd74101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.667+0000 7fbd79000640 1 -- 172.21.15.3:0/3670453226 shutdown_connections 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.667+0000 7fbd79000640 1 -- 172.21.15.3:0/3670453226 wait complete. 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.668+0000 7fbd79000640 1 Processor -- start 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.668+0000 7fbd79000640 1 -- start start 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd79000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 0x7fbd7419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd79000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd741042a0 0x7fbd7419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd79000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74109900 0x7fbd7419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:21.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd79000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74113d50 con 0x7fbd741042a0 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd79000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74113bd0 con 0x7fbd74103900 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd79000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74113ed0 con 0x7fbd74109900 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd73400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 0x7fbd7419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd73400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 0x7fbd7419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39484/0 (socket says 172.21.15.3:39484) 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd73400640 1 -- 172.21.15.3:0/2457348056 learned_addr learned my addr 172.21.15.3:0/2457348056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.669+0000 7fbd72a00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd741042a0 0x7fbd7419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73400640 1 -- 172.21.15.3:0/2457348056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74109900 msgr2=0x7fbd7419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73e00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74109900 0x7fbd7419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73400640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74109900 0x7fbd7419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73400640 1 -- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd741042a0 msgr2=0x7fbd7419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73400640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd741042a0 0x7fbd7419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:21.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73400640 1 -- 172.21.15.3:0/2457348056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7419f1b0 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73e00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74109900 0x7fbd7419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd72a00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd741042a0 0x7fbd7419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.670+0000 7fbd73400640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 0x7fbd7419a200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400e9f0 tx=0x7fbd6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.671+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400cd80 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.671+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64010910 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.671+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7419f4a0 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.671+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64018990 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.671+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a6d80 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.673+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd74103d60 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.676+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7fbd64010ac0 con 0x7fbd74103900 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.678+0000 7fbd6be00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbd580a2f20 0x7fbd580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:21.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.678+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbd640cfaa0 con 0x7fbd74103900 2024-08-01T04:17:21.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.678+0000 7fbd72a00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbd580a2f20 0x7fbd580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:21.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.680+0000 7fbd72a00640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbd580a2f20 0x7fbd580a53e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd60006010 tx=0x7fbd60005a20 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:21.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:21.681+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbd6409c440 con 0x7fbd74103900 2024-08-01T04:17:22.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.089+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd7410bef0 con 0x7fbd74103900 2024-08-01T04:17:22.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.090+0000 7fbd6be00640 1 -- 172.21.15.3:0/2457348056 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7fbd6409bd40 con 0x7fbd74103900 2024-08-01T04:17:22.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:17:22.093 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:17:22.093 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbd580a2f20 msgr2=0x7fbd580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbd580a2f20 0x7fbd580a53e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd60006010 tx=0x7fbd60005a20 comp rx=0 tx=0).stop 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 msgr2=0x7fbd7419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 0x7fbd7419a200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400e9f0 tx=0x7fbd6400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 shutdown_connections 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbd580a2f20 0x7fbd580a53e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74109900 0x7fbd7419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd741042a0 0x7fbd7419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 --2- 172.21.15.3:0/2457348056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd74103900 0x7fbd7419a200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 >> 172.21.15.3:0/2457348056 conn(0x7fbd740ff640 msgr2=0x7fbd740ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.094+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 shutdown_connections 2024-08-01T04:17:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:22.095+0000 7fbd79000640 1 -- 172.21.15.3:0/2457348056 wait complete. 2024-08-01T04:17:22.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:17:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:23 smithi003 ceph-mon[32262]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:17:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:23 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2457348056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:23.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:23 smithi160 ceph-mon[37224]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:17:23.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:23 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2457348056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:23.415 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:23.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:23 smithi163 ceph-mon[37383]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:17:23.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:23 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2457348056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:24.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 -- 172.21.15.3:0/1930399877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 msgr2=0x7f8a3410e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 --2- 172.21.15.3:0/1930399877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a3410e5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009c10 tx=0x7f8a2c02a6f0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 -- 172.21.15.3:0/1930399877 shutdown_connections 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 --2- 172.21.15.3:0/1930399877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a3410e5a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 --2- 172.21.15.3:0/1930399877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a341016d0 0x7f8a34101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 --2- 172.21.15.3:0/1930399877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a341076d0 0x7f8a34107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 -- 172.21.15.3:0/1930399877 >> 172.21.15.3:0/1930399877 conn(0x7f8a340fd430 msgr2=0x7f8a340ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 -- 172.21.15.3:0/1930399877 shutdown_connections 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.079+0000 7f8a33e00640 1 -- 172.21.15.3:0/1930399877 wait complete. 2024-08-01T04:17:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.080+0000 7f8a33e00640 1 Processor -- start 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 -- start start 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a341016d0 0x7f8a34197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a341984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a341076d0 0x7f8a3419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34111ad0 con 0x7f8a34102070 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34111950 con 0x7f8a341016d0 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a33e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34111c50 con 0x7f8a341076d0 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a341016d0 0x7f8a34197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.081+0000 7f8a32000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a341984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a341984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47996/0 (socket says 172.21.15.3:47996) 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 -- 172.21.15.3:0/3633021958 learned_addr learned my addr 172.21.15.3:0/3633021958 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a33400640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a341076d0 0x7f8a3419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 -- 172.21.15.3:0/3633021958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a341076d0 msgr2=0x7f8a3419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a341076d0 0x7f8a3419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 -- 172.21.15.3:0/3633021958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a341016d0 msgr2=0x7f8a34197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a341016d0 0x7f8a34197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 -- 172.21.15.3:0/3633021958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3419cf20 con 0x7f8a34102070 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.082+0000 7f8a32000640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a341984b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e9f0 tx=0x7f8a2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.083+0000 7f8a2b400640 1 -- 172.21.15.3:0/3633021958 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000ccc0 con 0x7f8a34102070 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.083+0000 7f8a2b400640 1 -- 172.21.15.3:0/3633021958 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a20010910 con 0x7f8a34102070 2024-08-01T04:17:24.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.083+0000 7f8a2b400640 1 -- 172.21.15.3:0/3633021958 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200188a0 con 0x7f8a34102070 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.083+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3419d210 con 0x7f8a34102070 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.083+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a34074150 con 0x7f8a34102070 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.088+0000 7f8a2b400640 1 -- 172.21.15.3:0/3633021958 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f8a20010ac0 con 0x7f8a34102070 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.088+0000 7f8a3c3c6740 1 Processor -- start 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 -- start start 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41dd0d0 0x55dfb41df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dfb412ebf0 con 0x55dfb41c9f20 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dfb40c6b20 con 0x55dfb41dd0d0 2024-08-01T04:17:24.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.089+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55dfb412abb0 con 0x55dfb41c9600 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a32000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41dd0d0 0x55dfb41df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a33400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a33400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54706/0 (socket says 172.21.15.3:54706) 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a32000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41dd0d0 0x55dfb41df4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:49894/0 (socket says 172.21.15.3:49894) 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a33400640 1 -- 172.21.15.3:0/1330374989 learned_addr learned my addr 172.21.15.3:0/1330374989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a2b400640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8a180a2f20 0x7f8a180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a2b400640 1 -- 172.21.15.3:0/3633021958 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8a200cfe10 con 0x7f8a34102070 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a33400640 1 -- 172.21.15.3:0/1330374989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41dd0d0 msgr2=0x55dfb41df4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a33400640 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41dd0d0 0x55dfb41df4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.090+0000 7f8a33400640 1 -- 172.21.15.3:0/1330374989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 msgr2=0x55dfb41dcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.091+0000 7f8a32a00640 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.091+0000 7f8a33400640 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41dcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.091+0000 7f8a33400640 1 -- 172.21.15.3:0/1330374989 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb40cd7f0 con 0x55dfb41c9600 2024-08-01T04:17:24.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.091+0000 7f8a32a00640 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.091+0000 7f8a33400640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8a180a2f20 0x7f8a180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.092+0000 7f8a33400640 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41c99e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c030b30 tx=0x7f8a2c030b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc46800abfea169e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.092+0000 7f8a1fe00640 1 -- 172.21.15.3:0/1330374989 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c036070 con 0x55dfb41c9600 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.092+0000 7f8a1fe00640 1 -- 172.21.15.3:0/1330374989 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a2c037040 con 0x55dfb41c9600 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.092+0000 7f8a1fe00640 1 -- 172.21.15.3:0/1330374989 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0047b0 con 0x55dfb41c9600 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/1330374989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 msgr2=0x55dfb41c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41c99e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c030b30 tx=0x7f8a2c030b60 comp rx=0 tx=0).stop 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a33400640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8a180a2f20 0x7f8a180a53e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c045000 tx=0x7f8a2c046000 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/1330374989 shutdown_connections 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41dd0d0 0x55dfb41df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/1330374989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41c99e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/1330374989 >> 172.21.15.3:0/1330374989 conn(0x55dfb41c5c70 msgr2=0x55dfb41c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.093+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/1330374989 shutdown_connections 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.094+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/1330374989 wait complete. 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.094+0000 7f8a3c3c6740 1 -- start start 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.094+0000 7f8a3c3c6740 1 Processor -- start 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41dd0d0 0x55dfb41daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dfb41dfc40 con 0x55dfb41dd0d0 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dfb40e9cf0 con 0x55dfb41c9f20 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55dfb41dfdc0 con 0x55dfb41c9600 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a3c3c6740 10 client.0 _ll_register_callbacks cb 0x55dfb40d5e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a33400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41dd0d0 0x55dfb41daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54722/0 (socket says 172.21.15.3:54722) 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a32a00640 1 -- 172.21.15.3:0/3586140622 learned_addr learned my addr 172.21.15.3:0/3586140622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.095+0000 7f8a32000640 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a32a00640 1 -- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 msgr2=0x55dfb41da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a32a00640 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a32a00640 1 -- 172.21.15.3:0/3586140622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41dd0d0 msgr2=0x55dfb41daad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a32a00640 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41dd0d0 0x55dfb41daad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a32a00640 1 -- 172.21.15.3:0/3586140622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb3ea03c0 con 0x55dfb41c9600 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a32a00640 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41da050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009880 tx=0x7f8a24009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=764dc88b001cdac6 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.096+0000 7f8a1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a1cc00640 1 -- 172.21.15.3:0/3586140622 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2400e8b0 con 0x55dfb41c9600 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a1cc00640 1 -- 172.21.15.3:0/3586140622 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8a2400ea50 con 0x55dfb41c9600 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c037040 con 0x55dfb41c9600 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a1cc00640 1 -- 172.21.15.3:0/3586140622 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a24026050 con 0x55dfb41c9600 2024-08-01T04:17:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a3c3c6740 10 client.25486 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dfb41eb9d0 con 0x55dfb41c9600 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.097+0000 7f8a1cc00640 1 -- 172.21.15.3:0/3586140622 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a2c037040 con 0x55dfb41c9600 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a1cc00640 1 -- 172.21.15.3:0/3586140622 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8a24021030 con 0x55dfb41c9600 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a3c3c6740 10 client.25486 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a3c3c6740 10 client.25486 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a3c3c6740 20 client.25486 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb41ece90 0x55dfb41ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a3c3c6740 4 client.25486 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb413c160 con 0x55dfb41ece90 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.098+0000 7f8a32000640 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb41ece90 0x55dfb41ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.099+0000 7f8a32000640 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb41ece90 0x55dfb41ef280 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa7c89be22801fa4 server_cookie=ad7a2089a3f68361 in_seq=0 out_seq=0 2024-08-01T04:17:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.099+0000 7f8a1cc00640 10 client.25486 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a1cc00640 1 -- 172.21.15.3:0/3586140622 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dfb413c160 con 0x55dfb41ece90 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a1cc00640 10 client.25486 handle_command_reply: tid=0 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 client.25486 shutdown 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 msgr2=0x55dfb41da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41da050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009880 tx=0x7f8a24009d40 comp rx=0 tx=0).stop 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 shutdown_connections 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41dd0d0 0x55dfb41daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb41ece90 0x55dfb41ef280 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3586140622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41c9600 0x55dfb41da050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 >> 172.21.15.3:0/3586140622 conn(0x55dfb41c5c70 msgr2=0x55dfb41cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.100+0000 7f8a1cc00640 0 client.25486 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.101+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 shutdown_connections 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.101+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3586140622 wait complete. 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.101+0000 7f8a3c3c6740 20 client.25486 trim_cache size 0 max 16384 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.128+0000 7f8a3c3c6740 1 Processor -- start 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.128+0000 7f8a3c3c6740 1 -- start start 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 0x55dfb41c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341adb70 con 0x55dfb41c9600 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341ab440 con 0x55dfb41c9f20 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341a9290 con 0x55dfb41dd0d0 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 0x55dfb41c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 0x55dfb41c0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48030/0 (socket says 172.21.15.3:48030) 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a32a00640 1 -- 172.21.15.3:0/3841691553 learned_addr learned my addr 172.21.15.3:0/3841691553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a32000640 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.129+0000 7f8a33400640 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32000640 1 -- 172.21.15.3:0/3841691553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 msgr2=0x55dfb41f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32000640 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32000640 1 -- 172.21.15.3:0/3841691553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 msgr2=0x55dfb41c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32000640 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 0x55dfb41c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32000640 1 -- 172.21.15.3:0/3841691553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb41d9000 con 0x55dfb41c9f20 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32a00640 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 0x55dfb41c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.130+0000 7f8a32000640 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41f4b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a200cd2e0 tx=0x7f8a20023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00d313604029391 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a1f400640 1 -- 172.21.15.3:0/3841691553 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2009c0b0 con 0x55dfb41c9f20 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a1f400640 1 -- 172.21.15.3:0/3841691553 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a200a0460 con 0x55dfb41c9f20 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a1f400640 1 -- 172.21.15.3:0/3841691553 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a200988b0 con 0x55dfb41c9f20 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3841691553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 msgr2=0x55dfb41f4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41f4b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a200cd2e0 tx=0x7f8a20023040 comp rx=0 tx=0).stop 2024-08-01T04:17:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3841691553 shutdown_connections 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9f20 0x55dfb41f4b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/3841691553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9600 0x55dfb41c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.131+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3841691553 >> 172.21.15.3:0/3841691553 conn(0x55dfb41c5c70 msgr2=0x55dfb41dc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.132+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3841691553 shutdown_connections 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.132+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/3841691553 wait complete. 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.132+0000 7f8a3c3c6740 1 -- start start 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.132+0000 7f8a3c3c6740 1 Processor -- start 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 0x55dfb41e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341a9900 con 0x55dfb41c9f20 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341aad70 con 0x55dfb41c9600 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341a9fd0 con 0x55dfb41dd0d0 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a3c3c6740 10 client.0 _ll_register_callbacks cb 0x7f8a24010db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 0x55dfb41e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 0x55dfb41e5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:49918/0 (socket says 172.21.15.3:49918) 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a32a00640 1 -- 172.21.15.3:0/787791550 learned_addr learned my addr 172.21.15.3:0/787791550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.133+0000 7f8a33400640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32000640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32a00640 1 -- 172.21.15.3:0/787791550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 msgr2=0x55dfb41e1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32a00640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41e1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32a00640 1 -- 172.21.15.3:0/787791550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 msgr2=0x55dfb41e0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32a00640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41e0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32a00640 1 -- 172.21.15.3:0/787791550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb41e19b0 con 0x55dfb41c9600 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a32a00640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 0x55dfb41e5860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a24019f50 tx=0x7f8a24010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5dafd9c2116ead2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.134+0000 7f8a1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a1cc00640 1 -- 172.21.15.3:0/787791550 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240052d0 con 0x55dfb41c9600 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a200a0460 con 0x55dfb41c9600 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a1cc00640 1 -- 172.21.15.3:0/787791550 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8a24005470 con 0x55dfb41c9600 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a1cc00640 1 -- 172.21.15.3:0/787791550 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a2403b050 con 0x55dfb41c9600 2024-08-01T04:17:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a3c3c6740 10 client.25505 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dfb4257bf0 con 0x55dfb41c9600 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.135+0000 7f8a1cc00640 1 -- 172.21.15.3:0/787791550 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a200a0460 con 0x55dfb41c9600 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a1cc00640 1 -- 172.21.15.3:0/787791550 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8a24021040 con 0x55dfb41c9600 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a3c3c6740 10 client.25505 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a3c3c6740 10 client.25505 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a3c3c6740 20 client.25505 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb42582e0 0x55dfb425a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a3c3c6740 4 client.25505 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfb4136670 con 0x55dfb42582e0 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.136+0000 7f8a32000640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb42582e0 0x55dfb425a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:24.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a32000640 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb42582e0 0x55dfb425a700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3169bbfd64010fc server_cookie=b29fb133d9d99972 in_seq=0 out_seq=0 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a1cc00640 10 client.25505 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a1cc00640 1 -- 172.21.15.3:0/787791550 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfb4136670 con 0x55dfb42582e0 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a1cc00640 10 client.25505 handle_command_reply: tid=0 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a3c3c6740 1 client.25505 shutdown 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 msgr2=0x55dfb41e5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.137+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 0x55dfb41e5860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a24019f50 tx=0x7f8a24010660 comp rx=0 tx=0).stop 2024-08-01T04:17:24.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 shutdown_connections 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55dfb42582e0 0x55dfb425a700 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dfb41dd0d0 0x55dfb41e1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dfb41c9f20 0x55dfb41e0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 --2- 172.21.15.3:0/787791550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dfb41c9600 0x55dfb41e5860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 >> 172.21.15.3:0/787791550 conn(0x55dfb41c5c70 msgr2=0x55dfb41c4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a1cc00640 0 client.25505 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:17:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 shutdown_connections 2024-08-01T04:17:24.190 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:17:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 1 -- 172.21.15.3:0/787791550 wait complete. 2024-08-01T04:17:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.138+0000 7f8a3c3c6740 20 client.25505 trim_cache size 0 max 16384 2024-08-01T04:17:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8a180a2f20 msgr2=0x7f8a180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8a180a2f20 0x7f8a180a53e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c045000 tx=0x7f8a2c046000 comp rx=0 tx=0).stop 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 msgr2=0x7f8a341984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a341984b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e9f0 tx=0x7f8a2000edf0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 shutdown_connections 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8a180a2f20 0x7f8a180a53e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a341076d0 0x7f8a3419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a34102070 0x7f8a341984b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 --2- 172.21.15.3:0/3633021958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a341016d0 0x7f8a34197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 >> 172.21.15.3:0/3633021958 conn(0x7f8a340fd430 msgr2=0x7f8a340fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.139+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 shutdown_connections 2024-08-01T04:17:24.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:24.140+0000 7f8a33e00640 1 -- 172.21.15.3:0/3633021958 wait complete. 2024-08-01T04:17:25.095 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:17:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:25 smithi003 ceph-mon[32262]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:25 smithi160 ceph-mon[37224]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:25.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:25 smithi163 ceph-mon[37383]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:25.578 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:26.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 -- 172.21.15.3:0/3104530654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f80074b60 msgr2=0x7f6f8010da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 --2- 172.21.15.3:0/3104530654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f80074b60 0x7f6f8010da80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009900 tx=0x7f6f6c02a480 comp rx=0 tx=0).stop 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 -- 172.21.15.3:0/3104530654 shutdown_connections 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 --2- 172.21.15.3:0/3104530654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f8010dfc0 0x7f6f801103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 --2- 172.21.15.3:0/3104530654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f80074b60 0x7f6f8010da80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 --2- 172.21.15.3:0/3104530654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074240 0x7f6f80074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.242+0000 7f6f86400640 1 -- 172.21.15.3:0/3104530654 >> 172.21.15.3:0/3104530654 conn(0x7f6f800fd440 msgr2=0x7f6f800ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.243+0000 7f6f86400640 1 -- 172.21.15.3:0/3104530654 shutdown_connections 2024-08-01T04:17:26.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.244+0000 7f6f86400640 1 -- 172.21.15.3:0/3104530654 wait complete. 2024-08-01T04:17:26.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.245+0000 7f6f86400640 1 Processor -- start 2024-08-01T04:17:26.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.245+0000 7f6f86400640 1 -- start start 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f86400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f80074240 0x7f6f8019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f86400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074b60 0x7f6f8019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f86400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8010dfc0 0x7f6f8019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f86400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80113d10 con 0x7f6f8010dfc0 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f86400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80113b90 con 0x7f6f80074240 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f86400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80113e90 con 0x7f6f80074b60 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074b60 0x7f6f8019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f85a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8010dfc0 0x7f6f8019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074b60 0x7f6f8019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54756/0 (socket says 172.21.15.3:54756) 2024-08-01T04:17:26.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.246+0000 7f6f7fe00640 1 -- 172.21.15.3:0/3224559892 learned_addr learned my addr 172.21.15.3:0/3224559892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:26.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85a00640 1 -- 172.21.15.3:0/3224559892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074b60 msgr2=0x7f6f8019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85a00640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074b60 0x7f6f8019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85000640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f80074240 0x7f6f8019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85a00640 1 -- 172.21.15.3:0/3224559892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f80074240 msgr2=0x7f6f8019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85a00640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f80074240 0x7f6f8019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85a00640 1 -- 172.21.15.3:0/3224559892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019f200 con 0x7f6f8010dfc0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85000640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f80074240 0x7f6f8019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.247+0000 7f6f85a00640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8010dfc0 0x7f6f8019eb20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005fd0 tx=0x7f6f7800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.248+0000 7f6f7d600640 1 -- 172.21.15.3:0/3224559892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7800cdf0 con 0x7f6f8010dfc0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.248+0000 7f6f7d600640 1 -- 172.21.15.3:0/3224559892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f78005020 con 0x7f6f8010dfc0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.248+0000 7f6f7d600640 1 -- 172.21.15.3:0/3224559892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f780106d0 con 0x7f6f8010dfc0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.248+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019f4f0 con 0x7f6f8010dfc0 2024-08-01T04:17:26.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.248+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801a6dd0 con 0x7f6f8010dfc0 2024-08-01T04:17:26.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.252+0000 7f6f8a439740 1 Processor -- start 2024-08-01T04:17:26.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.253+0000 7f6f7d600640 1 -- 172.21.15.3:0/3224559892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f6f780054d0 con 0x7f6f8010dfc0 2024-08-01T04:17:26.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 -- start start 2024-08-01T04:17:26.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e36902380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e369028c0 0x562e36904cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562e3685cbf0 con 0x562e368f8000 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562e367f4b20 con 0x562e368f76e0 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.254+0000 7f6f8a439740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562e36858bb0 con 0x562e369028c0 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:49938/0 (socket says 172.21.15.3:49938) 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 -- 172.21.15.3:0/2344237132 learned_addr learned my addr 172.21.15.3:0/2344237132 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:26.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e369028c0 0x562e36904cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f85000640 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e36902380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 -- 172.21.15.3:0/2344237132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 msgr2=0x562e368f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 -- 172.21.15.3:0/2344237132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 msgr2=0x562e36902380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e36902380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.255+0000 7f6f7fe00640 1 -- 172.21.15.3:0/2344237132 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e367fb7f0 con 0x562e369028c0 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.256+0000 7f6f85000640 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e36902380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.256+0000 7f6f7d600640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6f680a2f20 0x7f6f680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.256+0000 7f6f7d600640 1 -- 172.21.15.3:0/3224559892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6f780cfde0 con 0x7f6f8010dfc0 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.256+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e369028c0 0x562e36904cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c006010 tx=0x7f6f6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a5f1b1c0a85a8d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.256+0000 7f6f85000640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6f680a2f20 0x7f6f680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.256+0000 7f6f72000640 1 -- 172.21.15.3:0/2344237132 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c041070 con 0x562e369028c0 2024-08-01T04:17:26.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.257+0000 7f6f72000640 1 -- 172.21.15.3:0/2344237132 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c009520 con 0x562e369028c0 2024-08-01T04:17:26.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.257+0000 7f6f72000640 1 -- 172.21.15.3:0/2344237132 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c041070 con 0x562e369028c0 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 -- 172.21.15.3:0/2344237132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e369028c0 msgr2=0x562e36904cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e369028c0 0x562e36904cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c006010 tx=0x7f6f6c0046c0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f85000640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6f680a2f20 0x7f6f680a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x562e368f4ad0 tx=0x7f6f74005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 -- 172.21.15.3:0/2344237132 shutdown_connections 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e369028c0 0x562e36904cb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e36902380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2344237132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 -- 172.21.15.3:0/2344237132 >> 172.21.15.3:0/2344237132 conn(0x562e368f3c70 msgr2=0x562e368f4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:26.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 -- 172.21.15.3:0/2344237132 shutdown_connections 2024-08-01T04:17:26.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.258+0000 7f6f8a439740 1 -- 172.21.15.3:0/2344237132 wait complete. 2024-08-01T04:17:26.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.259+0000 7f6f8a439740 1 -- start start 2024-08-01T04:17:26.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.259+0000 7f6f8a439740 1 Processor -- start 2024-08-01T04:17:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 0x562e368ff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e368ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e36900190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562e369054e0 con 0x562e368f8000 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562e36817cf0 con 0x562e369028c0 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562e36905660 con 0x562e368f76e0 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f8a439740 10 client.0 _ll_register_callbacks cb 0x562e36803e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f85000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 0x562e368ff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f85000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 0x562e368ff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54780/0 (socket says 172.21.15.3:54780) 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f85000640 1 -- 172.21.15.3:0/2918766942 learned_addr learned my addr 172.21.15.3:0/2918766942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f85a00640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e36900190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.260+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e368ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f85000640 1 -- 172.21.15.3:0/2918766942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 msgr2=0x562e36900190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f85000640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e36900190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f85000640 1 -- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 msgr2=0x562e368ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f85000640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e368ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f85000640 1 -- 172.21.15.3:0/2918766942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e365ce3c0 con 0x562e368f76e0 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f85000640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 0x562e368ff710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f740096c0 tx=0x7f6f740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce428935ab8fd7d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f66a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.261+0000 7f6f66a00640 1 -- 172.21.15.3:0/2918766942 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7400fce0 con 0x562e368f76e0 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.262+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6c009520 con 0x562e368f76e0 2024-08-01T04:17:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.262+0000 7f6f66a00640 1 -- 172.21.15.3:0/2918766942 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6f7401f420 con 0x562e368f76e0 2024-08-01T04:17:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.262+0000 7f6f66a00640 1 -- 172.21.15.3:0/2918766942 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f7400fe80 con 0x562e368f76e0 2024-08-01T04:17:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.262+0000 7f6f8a439740 10 client.25504 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.262+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e36911110 con 0x562e368f76e0 2024-08-01T04:17:26.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f66a00640 1 -- 172.21.15.3:0/2918766942 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c009520 con 0x562e368f76e0 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f66a00640 1 -- 172.21.15.3:0/2918766942 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6f740316c0 con 0x562e368f76e0 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f8a439740 10 client.25504 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f8a439740 10 client.25504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f8a439740 20 client.25504 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e369128b0 0x562e36914d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f8a439740 4 client.25504 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.263+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e3686a160 con 0x562e369128b0 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.264+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e369128b0 0x562e36914d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.264+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e369128b0 0x562e36914d30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eeeed09f45cdb3d4 server_cookie=3a4294a9e952b633 in_seq=0 out_seq=0 2024-08-01T04:17:26.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.264+0000 7f6f66a00640 10 client.25504 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:26.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f66a00640 1 -- 172.21.15.3:0/2918766942 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e3686a160 con 0x562e369128b0 2024-08-01T04:17:26.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f66a00640 10 client.25504 handle_command_reply: tid=0 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f8a439740 1 client.25504 shutdown 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 msgr2=0x562e368ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 0x562e368ff710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f740096c0 tx=0x7f6f740046c0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 shutdown_connections 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e369128b0 0x562e36914d30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.265+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e36900190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f8000 0x562e368ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f8a439740 1 --2- 172.21.15.3:0/2918766942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f76e0 0x562e368ff710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 >> 172.21.15.3:0/2918766942 conn(0x562e368f3c70 msgr2=0x562e368f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f66a00640 0 client.25504 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 shutdown_connections 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f8a439740 1 -- 172.21.15.3:0/2918766942 wait complete. 2024-08-01T04:17:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.266+0000 7f6f8a439740 20 client.25504 trim_cache size 0 max 16384 2024-08-01T04:17:26.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.280+0000 7f6f8a439740 1 Processor -- start 2024-08-01T04:17:26.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.280+0000 7f6f8a439740 1 -- start start 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3691a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e369028c0 0x562e3691ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f8a439740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801b2130 con 0x562e369028c0 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f8a439740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801afa00 con 0x562e368f76e0 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f8a439740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801ad7f0 con 0x562e368f8000 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3691a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f85000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f85a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e369028c0 0x562e3691ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f7fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3691a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54796/0 (socket says 172.21.15.3:54796) 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.281+0000 7f6f7fe00640 1 -- 172.21.15.3:0/1591292770 learned_addr learned my addr 172.21.15.3:0/1591292770 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f7fe00640 1 -- 172.21.15.3:0/1591292770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 msgr2=0x562e368fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f7fe00640 1 -- 172.21.15.3:0/1591292770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e369028c0 msgr2=0x562e3691ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e369028c0 0x562e3691ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f7fe00640 1 -- 172.21.15.3:0/1591292770 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e368f7e30 con 0x562e368f8000 2024-08-01T04:17:26.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3691a640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c004d90 tx=0x7f6f6c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6e8c1ee9aa156e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f71600640 1 -- 172.21.15.3:0/1591292770 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c05a070 con 0x562e368f8000 2024-08-01T04:17:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f71600640 1 -- 172.21.15.3:0/1591292770 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c005700 con 0x562e368f8000 2024-08-01T04:17:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.282+0000 7f6f71600640 1 -- 172.21.15.3:0/1591292770 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c041070 con 0x562e368f8000 2024-08-01T04:17:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 -- 172.21.15.3:0/1591292770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 msgr2=0x562e3691a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3691a640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c004d90 tx=0x7f6f6c009260 comp rx=0 tx=0).stop 2024-08-01T04:17:26.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 -- 172.21.15.3:0/1591292770 shutdown_connections 2024-08-01T04:17:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e369028c0 0x562e3691ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3691a640 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 --2- 172.21.15.3:0/1591292770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e368f76e0 0x562e368fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.283+0000 7f6f8a439740 1 -- 172.21.15.3:0/1591292770 >> 172.21.15.3:0/1591292770 conn(0x562e368f3c70 msgr2=0x562e368fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.284+0000 7f6f8a439740 1 -- 172.21.15.3:0/1591292770 shutdown_connections 2024-08-01T04:17:26.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.284+0000 7f6f8a439740 1 -- 172.21.15.3:0/1591292770 wait complete. 2024-08-01T04:17:26.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.284+0000 7f6f8a439740 1 -- start start 2024-08-01T04:17:26.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.285+0000 7f6f8a439740 1 Processor -- start 2024-08-01T04:17:26.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.285+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 0x562e3690a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.285+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3690adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f85000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 0x562e3690a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f8a439740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e368fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f85000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 0x562e3690a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48102/0 (socket says 172.21.15.3:48102) 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f85000640 1 -- 172.21.15.3:0/3540867620 learned_addr learned my addr 172.21.15.3:0/3540867620 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801adec0 con 0x562e368f76e0 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801af330 con 0x562e369028c0 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801ae590 con 0x562e368f8000 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3690adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f8a439740 10 client.0 _ll_register_callbacks cb 0x7f6f740128e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.286+0000 7f6f85a00640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e368fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f85000640 1 -- 172.21.15.3:0/3540867620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 msgr2=0x562e3690adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f85000640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3690adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f85000640 1 -- 172.21.15.3:0/3540867620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 msgr2=0x562e368fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f85000640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e368fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f85000640 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e368fbfc0 con 0x562e368f76e0 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f85000640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 0x562e3690a8b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005780 tx=0x7f6f740316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ac0a35b8f6f8a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.287+0000 7f6f66a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.288+0000 7f6f66a00640 1 -- 172.21.15.3:0/3540867620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74036040 con 0x562e368f76e0 2024-08-01T04:17:26.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.288+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6c005700 con 0x562e368f76e0 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.288+0000 7f6f66a00640 1 -- 172.21.15.3:0/3540867620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6f74004040 con 0x562e368f76e0 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.288+0000 7f6f66a00640 1 -- 172.21.15.3:0/3540867620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f74009e90 con 0x562e368f76e0 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.288+0000 7f6f8a439740 10 client.15384 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.288+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e3697cc30 con 0x562e368f76e0 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f66a00640 1 -- 172.21.15.3:0/3540867620 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c005700 con 0x562e368f76e0 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f66a00640 1 -- 172.21.15.3:0/3540867620 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6f7401fc20 con 0x562e368f76e0 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f8a439740 10 client.15384 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f8a439740 10 client.15384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f8a439740 20 client.15384 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f8a439740 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e3697d400 0x562e3697f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f8a439740 4 client.15384 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:17:26.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.289+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e36864670 con 0x562e3697d400 2024-08-01T04:17:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.290+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e3697d400 0x562e3697f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:26.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f7fe00640 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e3697d400 0x562e3697f820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2438fe56a2a070f5 server_cookie=f095724904b22d04 in_seq=0 out_seq=0 2024-08-01T04:17:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f66a00640 10 client.15384 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f66a00640 1 -- 172.21.15.3:0/3540867620 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e36864670 con 0x562e3697d400 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f66a00640 10 client.15384 handle_command_reply: tid=0 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f8a439740 1 client.15384 shutdown 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 msgr2=0x562e3690a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.292+0000 7f6f8a439740 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 0x562e3690a8b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005780 tx=0x7f6f740316d0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 shutdown_connections 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x562e3697d400 0x562e3697f820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e369028c0 0x562e368fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f66a00640 0 client.15384 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e368f8000 0x562e3690adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 --2- 172.21.15.3:0/3540867620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e368f76e0 0x562e3690a8b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 >> 172.21.15.3:0/3540867620 conn(0x562e368f3c70 msgr2=0x562e368f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:26.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 shutdown_connections 2024-08-01T04:17:26.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.293+0000 7f6f8a439740 1 -- 172.21.15.3:0/3540867620 wait complete. 2024-08-01T04:17:26.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.294+0000 7f6f8a439740 20 client.15384 trim_cache size 0 max 16384 2024-08-01T04:17:26.297 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:17:26.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6f680a2f20 msgr2=0x7f6f680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6f680a2f20 0x7f6f680a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x562e368f4ad0 tx=0x7f6f74005f90 comp rx=0 tx=0).stop 2024-08-01T04:17:26.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8010dfc0 msgr2=0x7f6f8019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8010dfc0 0x7f6f8019eb20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6f78005fd0 tx=0x7f6f7800ef10 comp rx=0 tx=0).stop 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 shutdown_connections 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8010dfc0 0x7f6f8019eb20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6f680a2f20 0x7f6f680a53e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f80074b60 0x7f6f8019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 --2- 172.21.15.3:0/3224559892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f80074240 0x7f6f8019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.295+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 >> 172.21.15.3:0/3224559892 conn(0x7f6f800fd440 msgr2=0x7f6f800fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.296+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 shutdown_connections 2024-08-01T04:17:26.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:26.296+0000 7f6f86400640 1 -- 172.21.15.3:0/3224559892 wait complete. 2024-08-01T04:17:27.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:17:27.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:27 smithi160 ceph-mon[37224]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:27.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:27 smithi163 ceph-mon[37383]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:27 smithi003 ceph-mon[32262]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:27.610 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.330+0000 7fec33e00640 1 -- 172.21.15.3:0/3201487135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 msgr2=0x7fec34114990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.330+0000 7fec33e00640 1 --2- 172.21.15.3:0/3201487135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec34114990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009be0 tx=0x7fec2c02a6c0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.331+0000 7fec33e00640 1 -- 172.21.15.3:0/3201487135 shutdown_connections 2024-08-01T04:17:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.331+0000 7fec33e00640 1 --2- 172.21.15.3:0/3201487135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec34114990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.331+0000 7fec33e00640 1 --2- 172.21.15.3:0/3201487135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec34076d70 0x7fec34112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.331+0000 7fec33e00640 1 --2- 172.21.15.3:0/3201487135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 0x7fec34076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.331+0000 7fec33e00640 1 -- 172.21.15.3:0/3201487135 >> 172.21.15.3:0/3201487135 conn(0x7fec340ff640 msgr2=0x7fec34101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.332+0000 7fec33e00640 1 -- 172.21.15.3:0/3201487135 shutdown_connections 2024-08-01T04:17:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.333+0000 7fec33e00640 1 -- 172.21.15.3:0/3201487135 wait complete. 2024-08-01T04:17:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.334+0000 7fec33e00640 1 Processor -- start 2024-08-01T04:17:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.334+0000 7fec33e00640 1 -- start start 2024-08-01T04:17:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 0x7fec34111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec34076d70 0x7fec3410ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec3410b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec341180e0 con 0x7fec34076450 2024-08-01T04:17:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fec34117f60 con 0x7fec341125a0 2024-08-01T04:17:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec34118260 con 0x7fec34076d70 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec3410b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec3410b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50002/0 (socket says 172.21.15.3:50002) 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec33400640 1 -- 172.21.15.3:0/1989104028 learned_addr learned my addr 172.21.15.3:0/1989104028 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec32000640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec34076d70 0x7fec3410ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec33400640 1 -- 172.21.15.3:0/1989104028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec34076d70 msgr2=0x7fec3410ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec33400640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec34076d70 0x7fec3410ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec33400640 1 -- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 msgr2=0x7fec34111bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.335+0000 7fec32a00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 0x7fec34111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec33400640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 0x7fec34111bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec33400640 1 -- 172.21.15.3:0/1989104028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3410b7b0 con 0x7fec341125a0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec32a00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 0x7fec34111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.336+0000 7fec33400640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec3410b1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02abe0 tx=0x7fec2c0305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.337+0000 7fec38a00640 1 -- 172.21.15.3:0/1989104028 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c041020 con 0x7fec341125a0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.337+0000 7fec38a00640 1 -- 172.21.15.3:0/1989104028 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec2c003e50 con 0x7fec341125a0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.337+0000 7fec38a00640 1 -- 172.21.15.3:0/1989104028 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c036040 con 0x7fec341125a0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.337+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3410ba40 con 0x7fec341125a0 2024-08-01T04:17:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.337+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3410f990 con 0x7fec341125a0 2024-08-01T04:17:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.342+0000 7fec3c87a740 1 Processor -- start 2024-08-01T04:17:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.342+0000 7fec38a00640 1 -- 172.21.15.3:0/1989104028 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7fec2c044020 con 0x7fec341125a0 2024-08-01T04:17:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 -- start start 2024-08-01T04:17:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c33ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c40cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557224b98bf0 con 0x557224c34000 2024-08-01T04:17:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557224b30b20 con 0x557224c336e0 2024-08-01T04:17:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.343+0000 7fec3c87a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557224b94bb0 con 0x557224c3e8c0 2024-08-01T04:17:28.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.344+0000 7fec38a00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fec180a2fa0 0x7fec180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.344+0000 7fec32000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c33ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.344+0000 7fec38a00640 1 -- 172.21.15.3:0/1989104028 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fec2c0efdf0 con 0x7fec341125a0 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.344+0000 7fec32000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c40cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.345+0000 7fec32000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c40cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54840/0 (socket says 172.21.15.3:54840) 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.345+0000 7fec32000640 1 -- 172.21.15.3:0/2575071664 learned_addr learned my addr 172.21.15.3:0/2575071664 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.345+0000 7fec32a00640 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.345+0000 7fec33400640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fec180a2fa0 0x7fec180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.345+0000 7fec32000640 1 -- 172.21.15.3:0/2575071664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 msgr2=0x557224c33ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec32000640 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c33ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec32000640 1 -- 172.21.15.3:0/2575071664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 msgr2=0x557224c3e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec32000640 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec32000640 1 -- 172.21.15.3:0/2575071664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557224b377f0 con 0x557224c3e8c0 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec32a00640 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec32000640 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c40cb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec20013560 tx=0x7fec20004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2d5dfd9036173f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.346+0000 7fec33400640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fec180a2fa0 0x7fec180a5460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0ed320 tx=0x7fec2c009280 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.347+0000 7fec1fe00640 1 -- 172.21.15.3:0/2575071664 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20010490 con 0x557224c3e8c0 2024-08-01T04:17:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.347+0000 7fec1fe00640 1 -- 172.21.15.3:0/2575071664 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec20010b10 con 0x557224c3e8c0 2024-08-01T04:17:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.347+0000 7fec1fe00640 1 -- 172.21.15.3:0/2575071664 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2001fa30 con 0x557224c3e8c0 2024-08-01T04:17:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 -- 172.21.15.3:0/2575071664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 msgr2=0x557224c40cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c40cb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec20013560 tx=0x7fec20004730 comp rx=0 tx=0).stop 2024-08-01T04:17:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 -- 172.21.15.3:0/2575071664 shutdown_connections 2024-08-01T04:17:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c40cb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2575071664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c33ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.348+0000 7fec3c87a740 1 -- 172.21.15.3:0/2575071664 >> 172.21.15.3:0/2575071664 conn(0x557224c2fc70 msgr2=0x557224c30050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.349+0000 7fec3c87a740 1 -- 172.21.15.3:0/2575071664 shutdown_connections 2024-08-01T04:17:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.349+0000 7fec3c87a740 1 -- 172.21.15.3:0/2575071664 wait complete. 2024-08-01T04:17:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.349+0000 7fec3c87a740 1 -- start start 2024-08-01T04:17:28.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 Processor -- start 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c3b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557224c41490 con 0x557224c34000 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557224b53cf0 con 0x557224c336e0 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557224c41610 con 0x557224c3e8c0 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.350+0000 7fec3c87a740 10 client.0 _ll_register_callbacks cb 0x557224b3fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:28.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c3b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c3b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50020/0 (socket says 172.21.15.3:50020) 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 -- 172.21.15.3:0/2662805162 learned_addr learned my addr 172.21.15.3:0/2662805162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 -- 172.21.15.3:0/2662805162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 msgr2=0x557224c3c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c3c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32a00640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 -- 172.21.15.3:0/2662805162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 msgr2=0x557224c3bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32000640 1 -- 172.21.15.3:0/2662805162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722490a3c0 con 0x557224c336e0 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.351+0000 7fec32a00640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.352+0000 7fec32000640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c3b7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec20005d90 tx=0x7fec2001f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824ef40391b2a4b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.352+0000 7fec1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.352+0000 7fec1cc00640 1 -- 172.21.15.3:0/2662805162 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec200106a0 con 0x557224c336e0 2024-08-01T04:17:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.352+0000 7fec1cc00640 1 -- 172.21.15.3:0/2662805162 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fec20010840 con 0x557224c336e0 2024-08-01T04:17:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.352+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec20010b10 con 0x557224c336e0 2024-08-01T04:17:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.352+0000 7fec1cc00640 1 -- 172.21.15.3:0/2662805162 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec20004020 con 0x557224c336e0 2024-08-01T04:17:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 10 client.25532 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557224c4d190 con 0x557224c336e0 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec1cc00640 1 -- 172.21.15.3:0/2662805162 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec20010b10 con 0x557224c336e0 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec1cc00640 1 -- 172.21.15.3:0/2662805162 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fec2001d020 con 0x557224c336e0 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 10 client.25532 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 10 client.25532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 20 client.25532 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224c4e540 0x557224c50930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 4 client.25532 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.353+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557224ba6160 con 0x557224c4e540 2024-08-01T04:17:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.354+0000 7fec32a00640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224c4e540 0x557224c50930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.355+0000 7fec32a00640 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224c4e540 0x557224c50930 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f782ca5779d59b server_cookie=187259a014eb22fa in_seq=0 out_seq=0 2024-08-01T04:17:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.355+0000 7fec1cc00640 10 client.25532 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.356+0000 7fec1cc00640 1 -- 172.21.15.3:0/2662805162 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557224ba6160 con 0x557224c4e540 2024-08-01T04:17:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.356+0000 7fec1cc00640 10 client.25532 handle_command_reply: tid=0 2024-08-01T04:17:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.356+0000 7fec3c87a740 1 client.25532 shutdown 2024-08-01T04:17:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 msgr2=0x557224c3b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c3b7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec20005d90 tx=0x7fec2001f610 comp rx=0 tx=0).stop 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 shutdown_connections 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224c4e540 0x557224c50930 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c3e8c0 0x557224c3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2662805162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c336e0 0x557224c3b7f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 >> 172.21.15.3:0/2662805162 conn(0x557224c2fc70 msgr2=0x557224c359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.357+0000 7fec1cc00640 0 client.25532 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.358+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 shutdown_connections 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.358+0000 7fec3c87a740 1 -- 172.21.15.3:0/2662805162 wait complete. 2024-08-01T04:17:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.358+0000 7fec3c87a740 20 client.25532 trim_cache size 0 max 16384 2024-08-01T04:17:28.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.372+0000 7fec3c87a740 1 Processor -- start 2024-08-01T04:17:28.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.372+0000 7fec3c87a740 1 -- start start 2024-08-01T04:17:28.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.372+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c55fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c56520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c3e8c0 0x557224c35140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec3c87a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec341bede0 con 0x557224c34000 2024-08-01T04:17:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec3c87a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fec341bc6b0 con 0x557224c3e8c0 2024-08-01T04:17:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec3c87a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec341ba4a0 con 0x557224c336e0 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec32a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c56520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec32a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c56520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48150/0 (socket says 172.21.15.3:48150) 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.373+0000 7fec32a00640 1 -- 172.21.15.3:0/662163239 learned_addr learned my addr 172.21.15.3:0/662163239 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32000640 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c55fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32a00640 1 -- 172.21.15.3:0/662163239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 msgr2=0x557224c55fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32a00640 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c55fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32000640 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c3e8c0 0x557224c35140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32a00640 1 -- 172.21.15.3:0/662163239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c3e8c0 msgr2=0x557224c35140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32a00640 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c3e8c0 0x557224c35140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32a00640 1 -- 172.21.15.3:0/662163239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557224c39d60 con 0x557224c34000 2024-08-01T04:17:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.374+0000 7fec32a00640 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c56520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec24009c30 tx=0x7fec24014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdc59f9efbea726 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.375+0000 7fec1f400640 1 -- 172.21.15.3:0/662163239 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec240140d0 con 0x557224c34000 2024-08-01T04:17:28.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.375+0000 7fec1f400640 1 -- 172.21.15.3:0/662163239 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec24014270 con 0x557224c34000 2024-08-01T04:17:28.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec1f400640 1 -- 172.21.15.3:0/662163239 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec240126b0 con 0x557224c34000 2024-08-01T04:17:28.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 -- 172.21.15.3:0/662163239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 msgr2=0x557224c56520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c56520 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec24009c30 tx=0x7fec24014840 comp rx=0 tx=0).stop 2024-08-01T04:17:28.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 -- 172.21.15.3:0/662163239 shutdown_connections 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c3e8c0 0x557224c35140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c34000 0x557224c56520 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 --2- 172.21.15.3:0/662163239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c55fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.376+0000 7fec3c87a740 1 -- 172.21.15.3:0/662163239 >> 172.21.15.3:0/662163239 conn(0x557224c2fc70 msgr2=0x557224c36b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.377+0000 7fec3c87a740 1 -- 172.21.15.3:0/662163239 shutdown_connections 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.377+0000 7fec3c87a740 1 -- 172.21.15.3:0/662163239 wait complete. 2024-08-01T04:17:28.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.378+0000 7fec3c87a740 1 -- start start 2024-08-01T04:17:28.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.378+0000 7fec3c87a740 1 Processor -- start 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c34000 0x557224c46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c3e8c0 0x557224c38020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec341bab70 con 0x557224c3e8c0 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fec341bbfe0 con 0x557224c34000 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fec341bb240 con 0x557224c336e0 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec3c87a740 10 client.0 _ll_register_callbacks cb 0x7fec20009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec32a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c34000 0x557224c46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec32a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c34000 0x557224c46f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50050/0 (socket says 172.21.15.3:50050) 2024-08-01T04:17:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec32a00640 1 -- 172.21.15.3:0/2135132547 learned_addr learned my addr 172.21.15.3:0/2135132547 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec32000640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c3e8c0 0x557224c38020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.379+0000 7fec32000640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec32000640 1 -- 172.21.15.3:0/2135132547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 msgr2=0x557224c46a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec32000640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c46a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec32000640 1 -- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c34000 msgr2=0x557224c46f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec32000640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c34000 0x557224c46f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec32000640 1 -- 172.21.15.3:0/2135132547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557224c38690 con 0x557224c3e8c0 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec32000640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c3e8c0 0x557224c38020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec200097a0 tx=0x7fec2001f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92914292b3119e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.380+0000 7fec1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec1cc00640 1 -- 172.21.15.3:0/2135132547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20023040 con 0x557224c3e8c0 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec1cc00640 1 -- 172.21.15.3:0/2135132547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fec20029040 con 0x557224c3e8c0 2024-08-01T04:17:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec24014270 con 0x557224c3e8c0 2024-08-01T04:17:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec1cc00640 1 -- 172.21.15.3:0/2135132547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec20044050 con 0x557224c3e8c0 2024-08-01T04:17:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec3c87a740 10 client.15396 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557224cc9920 con 0x557224c3e8c0 2024-08-01T04:17:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.381+0000 7fec1cc00640 1 -- 172.21.15.3:0/2135132547 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec24014270 con 0x557224c3e8c0 2024-08-01T04:17:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec1cc00640 1 -- 172.21.15.3:0/2135132547 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fec2001d070 con 0x557224c3e8c0 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec3c87a740 10 client.15396 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec3c87a740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec3c87a740 20 client.15396 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224cc9f20 0x557224ccc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec3c87a740 4 client.15396 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.382+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557224ba0670 con 0x557224cc9f20 2024-08-01T04:17:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.383+0000 7fec32a00640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224cc9f20 0x557224ccc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.383+0000 7fec1cc00640 10 client.15396 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.383+0000 7fec32a00640 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224cc9f20 0x557224ccc340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a504c9ff26dcbf5 server_cookie=a15a26bf45e5a2e8 in_seq=0 out_seq=0 2024-08-01T04:17:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.384+0000 7fec1cc00640 1 -- 172.21.15.3:0/2135132547 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557224ba0670 con 0x557224cc9f20 2024-08-01T04:17:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.384+0000 7fec1cc00640 10 client.15396 handle_command_reply: tid=0 2024-08-01T04:17:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.384+0000 7fec3c87a740 1 client.15396 shutdown 2024-08-01T04:17:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c3e8c0 msgr2=0x557224c38020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c3e8c0 0x557224c38020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec200097a0 tx=0x7fec2001f610 comp rx=0 tx=0).stop 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 shutdown_connections 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557224cc9f20 0x557224ccc340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557224c3e8c0 0x557224c38020 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec1cc00640 0 client.15396 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557224c34000 0x557224c46f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 --2- 172.21.15.3:0/2135132547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557224c336e0 0x557224c46a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.385+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 >> 172.21.15.3:0/2135132547 conn(0x557224c2fc70 msgr2=0x557224c3d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:28.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.386+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 shutdown_connections 2024-08-01T04:17:28.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.386+0000 7fec3c87a740 1 -- 172.21.15.3:0/2135132547 wait complete. 2024-08-01T04:17:28.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.386+0000 7fec3c87a740 20 client.15396 trim_cache size 0 max 16384 2024-08-01T04:17:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:17:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.387+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fec180a2fa0 msgr2=0x7fec180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.387+0000 7fec33e00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fec180a2fa0 0x7fec180a5460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0ed320 tx=0x7fec2c009280 comp rx=0 tx=0).stop 2024-08-01T04:17:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.387+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 msgr2=0x7fec3410b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.387+0000 7fec33e00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec3410b1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec2c02abe0 tx=0x7fec2c0305d0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 shutdown_connections 2024-08-01T04:17:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec341125a0 0x7fec3410b1a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fec180a2fa0 0x7fec180a5460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec34076d70 0x7fec3410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 --2- 172.21.15.3:0/1989104028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec34076450 0x7fec34111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 >> 172.21.15.3:0/1989104028 conn(0x7fec340ff640 msgr2=0x7fec34112d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:28.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.388+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 shutdown_connections 2024-08-01T04:17:28.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:28.389+0000 7fec33e00640 1 -- 172.21.15.3:0/1989104028 wait complete. 2024-08-01T04:17:29.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:29 smithi160 ceph-mon[37224]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:17:29.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:29 smithi163 ceph-mon[37383]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:17:29.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:29 smithi003 ceph-mon[32262]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-01T04:17:30.244 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:17:30.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:17:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:31.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:31.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:17:31.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.292 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.293 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:31 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:31.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:17:31.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:31 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 -- 172.21.15.3:0/2986942847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2940742f0 msgr2=0x7ff2940746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2986942847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2940742f0 0x7ff2940746d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff288009900 tx=0x7ff28802a480 comp rx=0 tx=0).stop 2024-08-01T04:17:31.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 -- 172.21.15.3:0/2986942847 shutdown_connections 2024-08-01T04:17:31.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2986942847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff29410df90 0x7ff294110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2986942847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2986942847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2940742f0 0x7ff2940746d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 -- 172.21.15.3:0/2986942847 >> 172.21.15.3:0/2986942847 conn(0x7ff2940fd470 msgr2=0x7ff2940ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:31.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 -- 172.21.15.3:0/2986942847 shutdown_connections 2024-08-01T04:17:31.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.463+0000 7ff29aa00640 1 -- 172.21.15.3:0/2986942847 wait complete. 2024-08-01T04:17:31.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 Processor -- start 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 -- start start 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940742f0 0x7ff29410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff29410df90 0x7ff29410aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff294113d30 con 0x7ff29410df90 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff294113bb0 con 0x7ff294074c10 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.465+0000 7ff29aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff294113eb0 con 0x7ff2940742f0 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff298c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff299600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940742f0 0x7ff29410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff298c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50064/0 (socket says 172.21.15.3:50064) 2024-08-01T04:17:31.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff298c00640 1 -- 172.21.15.3:0/2616324372 learned_addr learned my addr 172.21.15.3:0/2616324372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff299600640 1 -- 172.21.15.3:0/2616324372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 msgr2=0x7ff29410a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff299600640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff299600640 1 -- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff29410df90 msgr2=0x7ff29410aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff299600640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff29410df90 0x7ff29410aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.466+0000 7ff299600640 1 -- 172.21.15.3:0/2616324372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2941aaa50 con 0x7ff2940742f0 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.467+0000 7ff299600640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940742f0 0x7ff29410d8e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff28802a9a0 tx=0x7ff288003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.467+0000 7ff292000640 1 -- 172.21.15.3:0/2616324372 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28803f070 con 0x7ff2940742f0 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.467+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2941aac20 con 0x7ff2940742f0 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.467+0000 7ff292000640 1 -- 172.21.15.3:0/2616324372 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff288031c90 con 0x7ff2940742f0 2024-08-01T04:17:31.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.467+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2941ab010 con 0x7ff2940742f0 2024-08-01T04:17:31.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.467+0000 7ff292000640 1 -- 172.21.15.3:0/2616324372 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff288035030 con 0x7ff2940742f0 2024-08-01T04:17:31.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.468+0000 7ff298c00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:31.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.472+0000 7ff29e9ea740 1 Processor -- start 2024-08-01T04:17:31.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.472+0000 7ff292000640 1 -- 172.21.15.3:0/2616324372 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7ff288042070 con 0x7ff2940742f0 2024-08-01T04:17:31.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.472+0000 7ff29e9ea740 1 -- start start 2024-08-01T04:17:31.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016507a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c0165122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c016514c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c0164227e0 con 0x55c016512830 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016507a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c01646dac0 con 0x55c016507650 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.473+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c0164058c0 con 0x55c016507f70 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016507a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50074/0 (socket says 172.21.15.3:50074) 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 -- 172.21.15.3:0/2947502893 learned_addr learned my addr 172.21.15.3:0/2947502893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff298c00640 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c016514c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 -- 172.21.15.3:0/2947502893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 msgr2=0x55c0165122f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c0165122f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 -- 172.21.15.3:0/2947502893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 msgr2=0x55c016514c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c016514c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 -- 172.21.15.3:0/2947502893 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01640c760 con 0x55c016507650 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.474+0000 7ff29a000640 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016507a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009dd0 tx=0x7ff28c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3b802865cb38a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.475+0000 7ff292000640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff27c0a2f20 0x7ff27c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.475+0000 7ff286600640 1 -- 172.21.15.3:0/2947502893 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004b20 con 0x55c016507650 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.475+0000 7ff286600640 1 -- 172.21.15.3:0/2947502893 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c003e50 con 0x55c016507650 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.475+0000 7ff286600640 1 -- 172.21.15.3:0/2947502893 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c005020 con 0x55c016507650 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.476+0000 7ff292000640 1 -- 172.21.15.3:0/2616324372 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff2880b79e0 con 0x7ff2940742f0 2024-08-01T04:17:31.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.476+0000 7ff299600640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff27c0a2f20 0x7ff27c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.476+0000 7ff29e9ea740 1 -- 172.21.15.3:0/2947502893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 msgr2=0x55c016507a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.476+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016507a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009dd0 tx=0x7ff28c00e720 comp rx=0 tx=0).stop 2024-08-01T04:17:31.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 -- 172.21.15.3:0/2947502893 shutdown_connections 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c016514c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c0165122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/2947502893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016507a30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 -- 172.21.15.3:0/2947502893 >> 172.21.15.3:0/2947502893 conn(0x55c016503be0 msgr2=0x55c016503fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff299600640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff27c0a2f20 0x7ff27c0a53e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff2880ec600 tx=0x7ff28802a480 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 -- 172.21.15.3:0/2947502893 shutdown_connections 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.477+0000 7ff29e9ea740 1 -- 172.21.15.3:0/2947502893 wait complete. 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.478+0000 7ff29e9ea740 1 -- start start 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.478+0000 7ff29e9ea740 1 Processor -- start 2024-08-01T04:17:31.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 0x55c01650f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507f70 0x55c01650fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c0165100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c016428ae0 con 0x55c016512830 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c016469a80 con 0x55c016507f70 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c0165154a0 con 0x55c016507650 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29e9ea740 10 client.0 _ll_register_callbacks cb 0x55c01642a190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.479+0000 7ff29a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 0x55c01650f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.480+0000 7ff29a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 0x55c01650f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54898/0 (socket says 172.21.15.3:54898) 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.480+0000 7ff29a000640 1 -- 172.21.15.3:0/1079007749 learned_addr learned my addr 172.21.15.3:0/1079007749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:31.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.480+0000 7ff29a000640 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c0165100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.480+0000 7ff298c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507f70 0x55c01650fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff29a000640 1 -- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507f70 msgr2=0x55c01650fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff29a000640 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507f70 0x55c01650fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff29a000640 1 -- 172.21.15.3:0/1079007749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 msgr2=0x55c0165100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff29a000640 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c0165100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff29a000640 1 -- 172.21.15.3:0/1079007749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0161df6a0 con 0x55c016507650 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff29a000640 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 0x55c01650f640 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009560 tx=0x7ff28c0044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3daff40bdebf720b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.481+0000 7ff27aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.482+0000 7ff27aa00640 1 -- 172.21.15.3:0/1079007749 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c018430 con 0x55c016507650 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.482+0000 7ff27aa00640 1 -- 172.21.15.3:0/1079007749 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff28c0185d0 con 0x55c016507650 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.482+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c003e50 con 0x55c016507650 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.483+0000 7ff27aa00640 1 -- 172.21.15.3:0/1079007749 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff28c0187d0 con 0x55c016507650 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.483+0000 7ff29e9ea740 10 client.25534 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:31.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.483+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c016520f50 con 0x55c016507650 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.483+0000 7ff27aa00640 1 -- 172.21.15.3:0/1079007749 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c003e50 con 0x55c016507650 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff27aa00640 1 -- 172.21.15.3:0/1079007749 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff28c028bc0 con 0x55c016507650 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff29e9ea740 10 client.25534 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff29e9ea740 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff29e9ea740 20 client.25534 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c016522300 0x55c0165246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff29e9ea740 4 client.25534 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:31.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c01647b030 con 0x55c016522300 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.484+0000 7ff298c00640 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c016522300 0x55c0165246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.485+0000 7ff298c00640 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c016522300 0x55c0165246f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f354d7aac1824d server_cookie=6ec6a470ab92d3e4 in_seq=0 out_seq=0 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.485+0000 7ff27aa00640 10 client.25534 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.485+0000 7ff27aa00640 1 -- 172.21.15.3:0/1079007749 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c01647b030 con 0x55c016522300 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.485+0000 7ff27aa00640 10 client.25534 handle_command_reply: tid=0 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 client.25534 shutdown 2024-08-01T04:17:31.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 msgr2=0x55c01650f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 0x55c01650f640 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009560 tx=0x7ff28c0044c0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 shutdown_connections 2024-08-01T04:17:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c016522300 0x55c0165246f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c0165100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507f70 0x55c01650fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/1079007749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507650 0x55c01650f640 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 >> 172.21.15.3:0/1079007749 conn(0x55c016503be0 msgr2=0x55c016502d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff27aa00640 0 client.25534 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.486+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 shutdown_connections 2024-08-01T04:17:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.487+0000 7ff29e9ea740 1 -- 172.21.15.3:0/1079007749 wait complete. 2024-08-01T04:17:31.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.487+0000 7ff29e9ea740 20 client.25534 trim_cache size 0 max 16384 2024-08-01T04:17:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.518+0000 7ff29e9ea740 1 Processor -- start 2024-08-01T04:17:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.518+0000 7ff29e9ea740 1 -- start start 2024-08-01T04:17:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.519+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016529de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.519+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c01652a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.519+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c01650dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.519+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff274011190 con 0x55c016512830 2024-08-01T04:17:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.519+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400ea60 con 0x55c016507650 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.519+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400c8b0 con 0x55c016507f70 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.520+0000 7ff29a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016529de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.520+0000 7ff29a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016529de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50082/0 (socket says 172.21.15.3:50082) 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.520+0000 7ff29a000640 1 -- 172.21.15.3:0/3365679351 learned_addr learned my addr 172.21.15.3:0/3365679351 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.520+0000 7ff29a000640 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c01650dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.520+0000 7ff298c00640 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c01652a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff29a000640 1 -- 172.21.15.3:0/3365679351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 msgr2=0x55c01652a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff29a000640 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c01652a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff29a000640 1 -- 172.21.15.3:0/3365679351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 msgr2=0x55c01650dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff29a000640 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c01650dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff29a000640 1 -- 172.21.15.3:0/3365679351 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0165073d0 con 0x55c016507650 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff29a000640 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016529de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0314a0 tx=0x7ff28c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e726dee6354fdfb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.521+0000 7ff285c00640 1 -- 172.21.15.3:0/3365679351 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c018430 con 0x55c016507650 2024-08-01T04:17:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.522+0000 7ff285c00640 1 -- 172.21.15.3:0/3365679351 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c037070 con 0x55c016507650 2024-08-01T04:17:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.522+0000 7ff285c00640 1 -- 172.21.15.3:0/3365679351 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0318a0 con 0x55c016507650 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 -- 172.21.15.3:0/3365679351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 msgr2=0x55c016529de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016529de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0314a0 tx=0x7ff28c00e720 comp rx=0 tx=0).stop 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 -- 172.21.15.3:0/3365679351 shutdown_connections 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016512830 0x55c01650dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016507f70 0x55c01652a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/3365679351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c016529de0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 -- 172.21.15.3:0/3365679351 >> 172.21.15.3:0/3365679351 conn(0x55c016503be0 msgr2=0x55c0165116c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 -- 172.21.15.3:0/3365679351 shutdown_connections 2024-08-01T04:17:31.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.523+0000 7ff29e9ea740 1 -- 172.21.15.3:0/3365679351 wait complete. 2024-08-01T04:17:31.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.524+0000 7ff29e9ea740 1 -- start start 2024-08-01T04:17:31.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.524+0000 7ff29e9ea740 1 Processor -- start 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c01651ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016507f70 0x55c016515fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016512830 0x55c016516500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400cf20 con 0x55c016507f70 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400e390 con 0x55c016507650 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400d5f0 con 0x55c016512830 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29e9ea740 10 client.0 _ll_register_callbacks cb 0x7ff28c003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c01651ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c01651ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50086/0 (socket says 172.21.15.3:50086) 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29a000640 1 -- 172.21.15.3:0/227347192 learned_addr learned my addr 172.21.15.3:0/227347192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff29a000640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016512830 0x55c016516500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.525+0000 7ff298c00640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016507f70 0x55c016515fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff29a000640 1 -- 172.21.15.3:0/227347192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016512830 msgr2=0x55c016516500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff29a000640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016512830 0x55c016516500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff29a000640 1 -- 172.21.15.3:0/227347192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016507f70 msgr2=0x55c016515fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff29a000640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016507f70 0x55c016515fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff29a000640 1 -- 172.21.15.3:0/227347192 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c016516af0 con 0x55c016507650 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff298c00640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016507f70 0x55c016515fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff29a000640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c01651ac90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0317a0 tx=0x7ff28c018b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a95b0aac8cb31e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.526+0000 7ff27aa00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:31.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.527+0000 7ff27aa00640 1 -- 172.21.15.3:0/227347192 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004010 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.527+0000 7ff27aa00640 1 -- 172.21.15.3:0/227347192 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff28c049040 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.527+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c037070 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.528+0000 7ff27aa00640 1 -- 172.21.15.3:0/227347192 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff28c0043a0 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.528+0000 7ff29e9ea740 10 client.25550 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.528+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c01659dc00 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.528+0000 7ff27aa00640 1 -- 172.21.15.3:0/227347192 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c037070 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff27aa00640 1 -- 172.21.15.3:0/227347192 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff28c028430 con 0x55c016507650 2024-08-01T04:17:31.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff29e9ea740 10 client.25550 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff29e9ea740 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff29e9ea740 20 client.25550 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c01659e260 0x55c0165a0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff29e9ea740 4 client.25550 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.529+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c0164fee50 con 0x55c01659e260 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.530+0000 7ff298c00640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c01659e260 0x55c0165a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.530+0000 7ff298c00640 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c01659e260 0x55c0165a0650 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7049f1cc4c5a4e6 server_cookie=1ec79a6689b5b800 in_seq=0 out_seq=0 2024-08-01T04:17:31.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.531+0000 7ff27aa00640 10 client.25550 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.532+0000 7ff27aa00640 1 -- 172.21.15.3:0/227347192 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0164fee50 con 0x55c01659e260 2024-08-01T04:17:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.532+0000 7ff27aa00640 10 client.25550 handle_command_reply: tid=0 2024-08-01T04:17:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.532+0000 7ff29e9ea740 1 client.25550 shutdown 2024-08-01T04:17:31.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.532+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 msgr2=0x55c01651ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.532+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c01651ac90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0317a0 tx=0x7ff28c018b10 comp rx=0 tx=0).stop 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 shutdown_connections 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c01659e260 0x55c0165a0650 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c016512830 0x55c016516500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c016507f70 0x55c016515fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff29e9ea740 1 --2- 172.21.15.3:0/227347192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c016507650 0x55c01651ac90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 >> 172.21.15.3:0/227347192 conn(0x55c016503be0 msgr2=0x55c01650dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.533+0000 7ff27aa00640 0 client.25550 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.534+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 shutdown_connections 2024-08-01T04:17:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.534+0000 7ff29e9ea740 1 -- 172.21.15.3:0/227347192 wait complete. 2024-08-01T04:17:31.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.534+0000 7ff29e9ea740 20 client.25550 trim_cache size 0 max 16384 2024-08-01T04:17:31.537 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:31.537 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:17:31.537 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "4dbab33a-80df-4904-8db5-fb48b32fc7ca", 2024-08-01T04:17:31.537 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:17:31.537 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.535+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff27c0a2f20 msgr2=0x7ff27c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.535+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff27c0a2f20 0x7ff27c0a53e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff2880ec600 tx=0x7ff28802a480 comp rx=0 tx=0).stop 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.535+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940742f0 msgr2=0x7ff29410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.535+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940742f0 0x7ff29410d8e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff28802a9a0 tx=0x7ff288003b20 comp rx=0 tx=0).stop 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 shutdown_connections 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff29410df90 0x7ff29410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff294074c10 0x7ff29410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff27c0a2f20 0x7ff27c0a53e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 --2- 172.21.15.3:0/2616324372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2940742f0 0x7ff29410d8e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:31.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 >> 172.21.15.3:0/2616324372 conn(0x7ff2940fd470 msgr2=0x7ff2940fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 shutdown_connections 2024-08-01T04:17:31.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:31.536+0000 7ff29aa00640 1 -- 172.21.15.3:0/2616324372 wait complete. 2024-08-01T04:17:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-01T04:17:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:17:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:31.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:31 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:32.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:32 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:17:32.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:32 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:17:32.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:32 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:32.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:32 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:17:32.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:32 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:17:32.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:32 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:32.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:17:32.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:32 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:17:32.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:32 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:17:32.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:32 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:17:33.025 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:33.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:33 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:17:33.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:33 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:33.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:33 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:17:33.369 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:33 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:33.370 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:33 smithi003 ceph-mon[32262]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-01T04:17:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:33 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:17:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:33 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:33 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:17:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:33 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:33 smithi160 ceph-mon[37224]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-01T04:17:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:33 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:17:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:33 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:33 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:17:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:33 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:17:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:33 smithi163 ceph-mon[37383]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-08-01T04:17:33.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 -- 172.21.15.3:0/1069563649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 msgr2=0x7f8f9c10e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 --2- 172.21.15.3:0/1069563649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c10e5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009cd0 tx=0x7f8f9402a8f0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 -- 172.21.15.3:0/1069563649 shutdown_connections 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 --2- 172.21.15.3:0/1069563649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c10e5c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 --2- 172.21.15.3:0/1069563649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 --2- 172.21.15.3:0/1069563649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c1076f0 0x7f8f9c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.675+0000 7f8fa3000640 1 -- 172.21.15.3:0/1069563649 >> 172.21.15.3:0/1069563649 conn(0x7f8f9c0fd470 msgr2=0x7f8f9c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.676+0000 7f8fa3000640 1 -- 172.21.15.3:0/1069563649 shutdown_connections 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.676+0000 7f8fa3000640 1 -- 172.21.15.3:0/1069563649 wait complete. 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.677+0000 7f8fa3000640 1 Processor -- start 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.677+0000 7f8fa3000640 1 -- start start 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa3000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa3000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa3000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c1076f0 0x7f8f9c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa3000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c111b40 con 0x7f8f9c102090 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa3000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1119c0 con 0x7f8f9c1076f0 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa3000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c111cc0 con 0x7f8f9c1016f0 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55014/0 (socket says 172.21.15.3:55014) 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa1c00640 1 -- 172.21.15.3:0/738572159 learned_addr learned my addr 172.21.15.3:0/738572159 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.678+0000 7f8fa1200640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa2600640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c1076f0 0x7f8f9c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1c00640 1 -- 172.21.15.3:0/738572159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c1076f0 msgr2=0x7f8f9c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1c00640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c1076f0 0x7f8f9c074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1c00640 1 -- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 msgr2=0x7f8f9c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1c00640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1c00640 1 -- 172.21.15.3:0/738572159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c1a6480 con 0x7f8f9c1016f0 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1200640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.679+0000 7f8fa1c00640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c074220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ea60 tx=0x7f8f9000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.680+0000 7f8f9a000640 1 -- 172.21.15.3:0/738572159 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000cde0 con 0x7f8f9c1016f0 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.680+0000 7f8f9a000640 1 -- 172.21.15.3:0/738572159 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f900035f0 con 0x7f8f9c1016f0 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.680+0000 7f8f9a000640 1 -- 172.21.15.3:0/738572159 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90010670 con 0x7f8f9c1016f0 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.680+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c1a6770 con 0x7f8f9c1016f0 2024-08-01T04:17:33.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.680+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c1a6c60 con 0x7f8f9c1016f0 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.683+0000 7f8fa709f740 1 Processor -- start 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 -- start start 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff29970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff34230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff36b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56422fe86bf0 con 0x56422ff29eb0 2024-08-01T04:17:33.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56422fe1ebd0 con 0x56422ff29590 2024-08-01T04:17:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.684+0000 7f8fa709f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56422fe82bb0 con 0x56422ff34770 2024-08-01T04:17:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff29970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff29970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51544/0 (socket says 172.21.15.3:51544) 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 -- 172.21.15.3:0/1560873699 learned_addr learned my addr 172.21.15.3:0/1560873699 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa1200640 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff36b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 -- 172.21.15.3:0/1560873699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 msgr2=0x56422ff36b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff36b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 -- 172.21.15.3:0/1560873699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 msgr2=0x56422ff34230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff34230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8fa2600640 1 -- 172.21.15.3:0/1560873699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422fe258a0 con 0x56422ff29590 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.685+0000 7f8f9a000640 1 -- 172.21.15.3:0/738572159 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f8f900037a0 con 0x7f8f9c1016f0 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.686+0000 7f8fa2600640 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff29970 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f94005e30 tx=0x7f8f94009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9368d235bfe0c85c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:33.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.687+0000 7f8f8ea00640 1 -- 172.21.15.3:0/1560873699 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94030620 con 0x56422ff29590 2024-08-01T04:17:33.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.687+0000 7f8f9a000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8f840a2f20 0x7f8f840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.687+0000 7f8f9a000640 1 -- 172.21.15.3:0/738572159 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8f900cfbb0 con 0x7f8f9c1016f0 2024-08-01T04:17:33.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.688+0000 7f8f8ea00640 1 -- 172.21.15.3:0/1560873699 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f94036070 con 0x56422ff29590 2024-08-01T04:17:33.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.688+0000 7f8f8ea00640 1 -- 172.21.15.3:0/1560873699 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94030620 con 0x56422ff29590 2024-08-01T04:17:33.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.688+0000 7f8fa1c00640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8f840a2f20 0x7f8f840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 -- 172.21.15.3:0/1560873699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 msgr2=0x56422ff29970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff29970 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f94005e30 tx=0x7f8f94009a20 comp rx=0 tx=0).stop 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa1c00640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8f840a2f20 0x7f8f840a53e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f900cd0e0 tx=0x7f8f900978f0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 -- 172.21.15.3:0/1560873699 shutdown_connections 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff36b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff34230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1560873699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff29970 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 -- 172.21.15.3:0/1560873699 >> 172.21.15.3:0/1560873699 conn(0x56422ff25b20 msgr2=0x56422ff25f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:33.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.689+0000 7f8fa709f740 1 -- 172.21.15.3:0/1560873699 shutdown_connections 2024-08-01T04:17:33.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.690+0000 7f8fa709f740 1 -- 172.21.15.3:0/1560873699 wait complete. 2024-08-01T04:17:33.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.690+0000 7f8fa709f740 1 -- start start 2024-08-01T04:17:33.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.690+0000 7f8fa709f740 1 Processor -- start 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 0x56422ff31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff32120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56422ff37340 con 0x56422ff29590 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56422fe41d40 con 0x56422ff29eb0 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56422ff374c0 con 0x56422ff34770 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa709f740 10 client.0 _ll_register_callbacks cb 0x56422fe0ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.691+0000 7f8fa1200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 0x56422ff31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 0x56422ff31be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51552/0 (socket says 172.21.15.3:51552) 2024-08-01T04:17:33.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 -- 172.21.15.3:0/4138179020 learned_addr learned my addr 172.21.15.3:0/4138179020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa2600640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff32120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa2600640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 -- 172.21.15.3:0/4138179020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 msgr2=0x56422ff32120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:33.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff32120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 -- 172.21.15.3:0/4138179020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 msgr2=0x56422ff316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa1200640 1 -- 172.21.15.3:0/4138179020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422fbf83c0 con 0x56422ff29eb0 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.692+0000 7f8fa2600640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.693+0000 7f8fa1200640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 0x56422ff31be0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f88013710 tx=0x7f8f88013b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e689097ac38a951 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.693+0000 7f8f83400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.693+0000 7f8f83400640 1 -- 172.21.15.3:0/4138179020 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011dd0 con 0x56422ff29eb0 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.693+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f94036070 con 0x56422ff29eb0 2024-08-01T04:17:33.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.693+0000 7f8f83400640 1 -- 172.21.15.3:0/4138179020 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8f88004020 con 0x56422ff29eb0 2024-08-01T04:17:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.694+0000 7f8f83400640 1 -- 172.21.15.3:0/4138179020 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f88004220 con 0x56422ff29eb0 2024-08-01T04:17:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.694+0000 7f8fa709f740 10 client.25559 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.694+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56422ff430a0 con 0x56422ff29eb0 2024-08-01T04:17:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8f83400640 1 -- 172.21.15.3:0/4138179020 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f94036070 con 0x56422ff29eb0 2024-08-01T04:17:33.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8f83400640 1 -- 172.21.15.3:0/4138179020 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8f8801a070 con 0x56422ff29eb0 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8fa709f740 10 client.25559 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8fa709f740 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8fa709f740 20 client.25559 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8fa709f740 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ff44550 0x56422ff469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8fa709f740 4 client.25559 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.695+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56422fe94160 con 0x56422ff44550 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.696+0000 7f8fa2600640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ff44550 0x56422ff469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.696+0000 7f8fa2600640 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ff44550 0x56422ff469d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e795794d6e4b7896 server_cookie=3a2dca76bfd82a60 in_seq=0 out_seq=0 2024-08-01T04:17:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.696+0000 7f8f83400640 10 client.25559 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.697+0000 7f8f83400640 1 -- 172.21.15.3:0/4138179020 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56422fe94160 con 0x56422ff44550 2024-08-01T04:17:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.697+0000 7f8f83400640 10 client.25559 handle_command_reply: tid=0 2024-08-01T04:17:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.697+0000 7f8fa709f740 1 client.25559 shutdown 2024-08-01T04:17:33.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 msgr2=0x56422ff31be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 0x56422ff31be0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f88013710 tx=0x7f8f88013b10 comp rx=0 tx=0).stop 2024-08-01T04:17:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 shutdown_connections 2024-08-01T04:17:33.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ff44550 0x56422ff469d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff32120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29eb0 0x56422ff31be0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 --2- 172.21.15.3:0/4138179020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 >> 172.21.15.3:0/4138179020 conn(0x56422ff25b20 msgr2=0x56422ff2b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8f83400640 0 client.25559 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 shutdown_connections 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.698+0000 7f8fa709f740 1 -- 172.21.15.3:0/4138179020 wait complete. 2024-08-01T04:17:33.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.699+0000 7f8fa709f740 20 client.25559 trim_cache size 0 max 16384 2024-08-01T04:17:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 Processor -- start 2024-08-01T04:17:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 -- start start 2024-08-01T04:17:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff2bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff29eb0 0x56422ff4c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff34770 0x56422ff4c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1add60 con 0x56422ff29590 2024-08-01T04:17:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1ab630 con 0x56422ff34770 2024-08-01T04:17:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.733+0000 7f8fa709f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1a94a0 con 0x56422ff29eb0 2024-08-01T04:17:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff2bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff2bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55574/0 (socket says 172.21.15.3:55574) 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa2600640 1 -- 172.21.15.3:0/1471897233 learned_addr learned my addr 172.21.15.3:0/1471897233 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa1200640 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff29eb0 0x56422ff4c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa1200640 1 -- 172.21.15.3:0/1471897233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff34770 msgr2=0x56422ff4c820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa1200640 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff34770 0x56422ff4c820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.734+0000 7f8fa1200640 1 -- 172.21.15.3:0/1471897233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 msgr2=0x56422ff2bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.735+0000 7f8fa1200640 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff2bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.735+0000 7f8fa1200640 1 -- 172.21.15.3:0/1471897233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422ff2c530 con 0x56422ff29eb0 2024-08-01T04:17:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.735+0000 7f8fa2600640 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff2bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.735+0000 7f8fa1200640 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff29eb0 0x56422ff4c2e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x56422ff32c50 tx=0x7f8f880117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb97c17281dee3ec server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.736+0000 7f8f8e000640 1 -- 172.21.15.3:0/1471897233 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88025470 con 0x56422ff29eb0 2024-08-01T04:17:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.736+0000 7f8f8e000640 1 -- 172.21.15.3:0/1471897233 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88005520 con 0x56422ff29eb0 2024-08-01T04:17:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.736+0000 7f8f8e000640 1 -- 172.21.15.3:0/1471897233 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88011ea0 con 0x56422ff29eb0 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 -- 172.21.15.3:0/1471897233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff29eb0 msgr2=0x56422ff4c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff29eb0 0x56422ff4c2e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x56422ff32c50 tx=0x7f8f880117b0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 -- 172.21.15.3:0/1471897233 shutdown_connections 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff34770 0x56422ff4c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff29eb0 0x56422ff4c2e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 --2- 172.21.15.3:0/1471897233 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29590 0x56422ff2bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 -- 172.21.15.3:0/1471897233 >> 172.21.15.3:0/1471897233 conn(0x56422ff25b20 msgr2=0x56422ff30790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 -- 172.21.15.3:0/1471897233 shutdown_connections 2024-08-01T04:17:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 -- 172.21.15.3:0/1471897233 wait complete. 2024-08-01T04:17:33.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.737+0000 7f8fa709f740 1 -- start start 2024-08-01T04:17:33.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.738+0000 7f8fa709f740 1 Processor -- start 2024-08-01T04:17:33.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.738+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff3c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.738+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff3cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.738+0000 7f8fa709f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff37c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa709f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1a9af0 con 0x56422ff29eb0 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa709f740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1aaf60 con 0x56422ff29590 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa709f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1aa1c0 con 0x56422ff34770 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa709f740 10 client.0 _ll_register_callbacks cb 0x7f8f88009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff3cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa2600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff37c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa2600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff37c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55042/0 (socket says 172.21.15.3:55042) 2024-08-01T04:17:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa2600640 1 -- 172.21.15.3:0/479613787 learned_addr learned my addr 172.21.15.3:0/479613787 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.739+0000 7f8fa2600640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff3c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa2600640 1 -- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 msgr2=0x56422ff3c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa2600640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff3c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa2600640 1 -- 172.21.15.3:0/479613787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 msgr2=0x56422ff3cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa2600640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff3cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa2600640 1 -- 172.21.15.3:0/479613787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422ff38480 con 0x56422ff34770 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa1200640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff3cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8fa2600640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff37c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8f94000f80 tx=0x7f8f9402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c4bbfce3133140 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.740+0000 7f8f83400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.741+0000 7f8f83400640 1 -- 172.21.15.3:0/479613787 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94049040 con 0x56422ff34770 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.741+0000 7f8f83400640 1 -- 172.21.15.3:0/479613787 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8f94030930 con 0x56422ff34770 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.741+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f88005520 con 0x56422ff34770 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.741+0000 7f8f83400640 1 -- 172.21.15.3:0/479613787 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f94003e20 con 0x56422ff34770 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.741+0000 7f8fa709f740 10 client.25555 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.741+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56422ffaed20 con 0x56422ff34770 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8f83400640 1 -- 172.21.15.3:0/479613787 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88005520 con 0x56422ff34770 2024-08-01T04:17:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8f83400640 1 -- 172.21.15.3:0/479613787 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8f94053450 con 0x56422ff34770 2024-08-01T04:17:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8fa709f740 10 client.25555 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8fa709f740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:17:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8fa709f740 20 client.25555 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8fa709f740 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ffaf060 0x56422ffb1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8fa709f740 4 client.25555 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:17:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.742+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56422fe8e670 con 0x56422ffaf060 2024-08-01T04:17:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.743+0000 7f8fa1200640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ffaf060 0x56422ffb1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.743+0000 7f8fa1200640 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ffaf060 0x56422ffb1450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe1bf467c24294c server_cookie=fc97139252b37514 in_seq=0 out_seq=0 2024-08-01T04:17:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.743+0000 7f8f83400640 10 client.25555 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.744+0000 7f8f83400640 1 -- 172.21.15.3:0/479613787 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56422fe8e670 con 0x56422ffaf060 2024-08-01T04:17:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.744+0000 7f8f83400640 10 client.25555 handle_command_reply: tid=0 2024-08-01T04:17:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.744+0000 7f8fa709f740 1 client.25555 shutdown 2024-08-01T04:17:33.749 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:33.749 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:17:33.749 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout: "4dbab33a-80df-4904-8db5-fb48b32fc7ca": { 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "4dbab33a-80df-4904-8db5-fb48b32fc7ca", 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:33.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 msgr2=0x56422ff37c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff37c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8f94000f80 tx=0x7f8f9402a8f0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 shutdown_connections 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56422ffaf060 0x56422ffb1450 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56422ff34770 0x56422ff37c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56422ff29eb0 0x56422ff3cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 --2- 172.21.15.3:0/479613787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56422ff29590 0x56422ff3c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 >> 172.21.15.3:0/479613787 conn(0x56422ff25b20 msgr2=0x56422ff31660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8f83400640 0 client.25555 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 shutdown_connections 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 1 -- 172.21.15.3:0/479613787 wait complete. 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.745+0000 7f8fa709f740 20 client.25555 trim_cache size 0 max 16384 2024-08-01T04:17:33.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8f840a2f20 msgr2=0x7f8f840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8f840a2f20 0x7f8f840a53e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f900cd0e0 tx=0x7f8f900978f0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 msgr2=0x7f8f9c074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c074220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ea60 tx=0x7f8f9000ef20 comp rx=0 tx=0).stop 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 shutdown_connections 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c1076f0 0x7f8f9c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f9c102090 0x7f8f9c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8f840a2f20 0x7f8f840a53e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 --2- 172.21.15.3:0/738572159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f9c1016f0 0x7f8f9c074220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.746+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 >> 172.21.15.3:0/738572159 conn(0x7f8f9c0fd470 msgr2=0x7f8f9c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.747+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 shutdown_connections 2024-08-01T04:17:33.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:33.747+0000 7f8fa3000640 1 -- 172.21.15.3:0/738572159 wait complete. 2024-08-01T04:17:34.615 INFO:tasks.cephfs.filesystem:scrub status for tag:4dbab33a-80df-4904-8db5-fb48b32fc7ca - {'path': '/', 'tag': '4dbab33a-80df-4904-8db5-fb48b32fc7ca', 'options': 'recursive,force'} 2024-08-01T04:17:35.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:35 smithi160 ceph-mon[37224]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:35.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:35 smithi163 ceph-mon[37383]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:35.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:35 smithi003 ceph-mon[32262]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:36.988 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:17:37.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:37 smithi160 ceph-mon[37224]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:37.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:37 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:17:37.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:37 smithi163 ceph-mon[37383]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:37.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:37 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:17:37.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:37 smithi003 ceph-mon[32262]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:37.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:37 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:17:37.650 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:38.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.308+0000 7f3d91c00640 1 -- 172.21.15.3:0/288228636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c074240 msgr2=0x7f3d8c074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:38.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.309+0000 7f3d91c00640 1 --2- 172.21.15.3:0/288228636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c074240 0x7f3d8c074620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009900 tx=0x7f3d7c02a480 comp rx=0 tx=0).stop 2024-08-01T04:17:38.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.309+0000 7f3d91c00640 1 -- 172.21.15.3:0/288228636 shutdown_connections 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.309+0000 7f3d91c00640 1 --2- 172.21.15.3:0/288228636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c10dfb0 0x7f3d8c1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.309+0000 7f3d91c00640 1 --2- 172.21.15.3:0/288228636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 0x7f3d8c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.309+0000 7f3d91c00640 1 --2- 172.21.15.3:0/288228636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c074240 0x7f3d8c074620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.309+0000 7f3d91c00640 1 -- 172.21.15.3:0/288228636 >> 172.21.15.3:0/288228636 conn(0x7f3d8c0fd470 msgr2=0x7f3d8c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.310+0000 7f3d91c00640 1 -- 172.21.15.3:0/288228636 shutdown_connections 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.310+0000 7f3d91c00640 1 -- 172.21.15.3:0/288228636 wait complete. 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 Processor -- start 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 -- start start 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c074240 0x7f3d8c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 0x7f3d8c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c10dfb0 0x7f3d8c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c113d00 con 0x7f3d8c074240 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c113b80 con 0x7f3d8c074b60 2024-08-01T04:17:38.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.312+0000 7f3d91c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c113e80 con 0x7f3d8c10dfb0 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.313+0000 7f3d8be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c074240 0x7f3d8c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.313+0000 7f3d8be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c074240 0x7f3d8c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55596/0 (socket says 172.21.15.3:55596) 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.313+0000 7f3d8be00640 1 -- 172.21.15.3:0/1224424437 learned_addr learned my addr 172.21.15.3:0/1224424437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.313+0000 7f3d91200640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c10dfb0 0x7f3d8c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.313+0000 7f3d8b400640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 0x7f3d8c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.314+0000 7f3d8b400640 1 -- 172.21.15.3:0/1224424437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c10dfb0 msgr2=0x7f3d8c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.314+0000 7f3d8b400640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c10dfb0 0x7f3d8c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.314+0000 7f3d8b400640 1 -- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c074240 msgr2=0x7f3d8c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.314+0000 7f3d8b400640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c074240 0x7f3d8c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.314+0000 7f3d8b400640 1 -- 172.21.15.3:0/1224424437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c19f230 con 0x7f3d8c074b60 2024-08-01T04:17:38.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.314+0000 7f3d8b400640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 0x7f3d8c19a7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800e9f0 tx=0x7f3d7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.315+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800ccc0 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.315+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d78010910 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.315+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c19f520 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.315+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c1a6e00 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.315+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78018960 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.317+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c005290 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.318+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f3d78010ac0 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.321+0000 7f3d88c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3d700a2f20 0x7f3d700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.321+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3d780cf8c0 con 0x7f3d8c074b60 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.322+0000 7f3d8be00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3d700a2f20 0x7f3d700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:38.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.323+0000 7f3d8be00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3d700a2f20 0x7f3d700a53e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0096c0 tx=0x7f3d7c0046c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:38.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.324+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3d7809bc10 con 0x7f3d8c074b60 2024-08-01T04:17:38.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.568+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d6c005850 con 0x7f3d8c074b60 2024-08-01T04:17:38.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.569+0000 7f3d88c00640 1 -- 172.21.15.3:0/1224424437 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7f3d7809ff10 con 0x7f3d8c074b60 2024-08-01T04:17:38.572 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:17:38.572 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:17:38.572 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:17:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3d700a2f20 msgr2=0x7f3d700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3d700a2f20 0x7f3d700a53e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0096c0 tx=0x7f3d7c0046c0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 msgr2=0x7f3d8c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 0x7f3d8c19a7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800e9f0 tx=0x7f3d7800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 shutdown_connections 2024-08-01T04:17:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3d700a2f20 0x7f3d700a53e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d8c10dfb0 0x7f3d8c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3d8c074b60 0x7f3d8c19a7c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 --2- 172.21.15.3:0/1224424437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d8c074240 0x7f3d8c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.572+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 >> 172.21.15.3:0/1224424437 conn(0x7f3d8c0fd470 msgr2=0x7f3d8c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:38.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.573+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 shutdown_connections 2024-08-01T04:17:38.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:38.573+0000 7f3d91c00640 1 -- 172.21.15.3:0/1224424437 wait complete. 2024-08-01T04:17:39.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:39 smithi003 ceph-mon[32262]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-01T04:17:39.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:39 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1224424437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:39 smithi160 ceph-mon[37224]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-01T04:17:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:39 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1224424437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:39.420 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db --set-id 5c87d78e --release --await' 2024-08-01T04:17:39.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db --set-id 5c87d78e --release --await 2024-08-01T04:17:39.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:39 smithi163 ceph-mon[37383]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-01T04:17:39.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:39 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1224424437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.561+0000 7f9590a00640 1 -- 172.21.15.3:0/2009113787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c074b60 msgr2=0x7f958c10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.561+0000 7f9590a00640 1 --2- 172.21.15.3:0/2009113787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c074b60 0x7f958c10da70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9578009900 tx=0x7f957802a480 comp rx=0 tx=0).stop 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 -- 172.21.15.3:0/2009113787 shutdown_connections 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 --2- 172.21.15.3:0/2009113787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c10dfb0 0x7f958c1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 --2- 172.21.15.3:0/2009113787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c074b60 0x7f958c10da70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 --2- 172.21.15.3:0/2009113787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f958c074240 0x7f958c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 -- 172.21.15.3:0/2009113787 >> 172.21.15.3:0/2009113787 conn(0x7f958c0fd470 msgr2=0x7f958c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 -- 172.21.15.3:0/2009113787 shutdown_connections 2024-08-01T04:17:40.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.562+0000 7f9590a00640 1 -- 172.21.15.3:0/2009113787 wait complete. 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.563+0000 7f9590a00640 1 Processor -- start 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.563+0000 7f9590a00640 1 -- start start 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.563+0000 7f9590a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 0x7f958c10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f9590a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f958c074b60 0x7f958c10aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f9590a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c10dfb0 0x7f958c10b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f9590a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f958c113ce0 con 0x7f958c10dfb0 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f9590a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f958c113b60 con 0x7f958c074240 2024-08-01T04:17:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f9590a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f958c113e60 con 0x7f958c074b60 2024-08-01T04:17:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 0x7f958c10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 0x7f958c10d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51600/0 (socket says 172.21.15.3:51600) 2024-08-01T04:17:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 -- 172.21.15.3:0/1710610617 learned_addr learned my addr 172.21.15.3:0/1710610617 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 -- 172.21.15.3:0/1710610617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f958c074b60 msgr2=0x7f958c10aa50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-01T04:17:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f958c074b60 0x7f958c10aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 -- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c10dfb0 msgr2=0x7f958c10b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c10dfb0 0x7f958c10b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.564+0000 7f958b400640 1 -- 172.21.15.3:0/1710610617 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c1aaa00 con 0x7f958c074240 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.565+0000 7f958b400640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 0x7f958c10d9a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f957c00e9f0 tx=0x7f957c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.565+0000 7f9583e00640 1 -- 172.21.15.3:0/1710610617 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c00cd80 con 0x7f958c074240 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.565+0000 7f9583e00640 1 -- 172.21.15.3:0/1710610617 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f957c005500 con 0x7f958c074240 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.565+0000 7f9583e00640 1 -- 172.21.15.3:0/1710610617 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c010690 con 0x7f958c074240 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.565+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c1aabd0 con 0x7f958c074240 2024-08-01T04:17:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.565+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c1ab060 con 0x7f958c074240 2024-08-01T04:17:40.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.568+0000 7f9594ac2740 1 Processor -- start 2024-08-01T04:17:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.568+0000 7f9594ac2740 1 -- start start 2024-08-01T04:17:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe85c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe866e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe8697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f9594ac2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5610fe7c1e30 con 0x5610fe8673d0 2024-08-01T04:17:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f9594ac2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5610fe759d40 con 0x5610fe85cb10 2024-08-01T04:17:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f9594ac2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5610fe7bddf0 con 0x5610fe85c1f0 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f958be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe85c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f958aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe8697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f958be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe85c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55090/0 (socket says 172.21.15.3:55090) 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.569+0000 7f958be00640 1 -- 172.21.15.3:0/1132391977 learned_addr learned my addr 172.21.15.3:0/1132391977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f958be00640 1 -- 172.21.15.3:0/1132391977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 msgr2=0x5610fe866e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f958be00640 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe866e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f958be00640 1 -- 172.21.15.3:0/1132391977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 msgr2=0x5610fe8697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f958be00640 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe8697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f958be00640 1 -- 172.21.15.3:0/1132391977 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fe4d1580 con 0x5610fe85c1f0 2024-08-01T04:17:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f958be00640 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe85c5d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9584009e00 tx=0x7f958400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ef4999e7d52e17f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f9577e00640 1 -- 172.21.15.3:0/1132391977 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584004d20 con 0x5610fe85c1f0 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f9577e00640 1 -- 172.21.15.3:0/1132391977 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9584003e50 con 0x5610fe85c1f0 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.570+0000 7f9577e00640 1 -- 172.21.15.3:0/1132391977 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584005020 con 0x5610fe85c1f0 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9583e00640 1 -- 172.21.15.3:0/1710610617 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f957c0046c0 con 0x7f958c074240 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 -- 172.21.15.3:0/1132391977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 msgr2=0x5610fe85c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe85c5d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9584009e00 tx=0x7f958400e720 comp rx=0 tx=0).stop 2024-08-01T04:17:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 -- 172.21.15.3:0/1132391977 shutdown_connections 2024-08-01T04:17:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe8697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe866e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 --2- 172.21.15.3:0/1132391977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe85c5d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.571+0000 7f9594ac2740 1 -- 172.21.15.3:0/1132391977 >> 172.21.15.3:0/1132391977 conn(0x5610fe858780 msgr2=0x5610fe858b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.572+0000 7f9594ac2740 1 -- 172.21.15.3:0/1132391977 shutdown_connections 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.572+0000 7f9594ac2740 1 -- 172.21.15.3:0/1132391977 wait complete. 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.572+0000 7f9594ac2740 1 -- start start 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.573+0000 7f9594ac2740 1 Processor -- start 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.573+0000 7f9583e00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95700a2db0 0x7f95700a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.573+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 0x5610fe864250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.573+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe864790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.573+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe8673d0 0x5610fe864cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.573+0000 7f9594ac2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5610fe869fa0 con 0x5610fe85cb10 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f9594ac2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5610fe77ce70 con 0x5610fe85c1f0 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f9594ac2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5610fe86a120 con 0x5610fe8673d0 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f9594ac2740 10 client.0 _ll_register_callbacks cb 0x5610fe76bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f9583e00640 1 -- 172.21.15.3:0/1710610617 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f957c0cfc90 con 0x7f958c074240 2024-08-01T04:17:40.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f958b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 0x5610fe864250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f958b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 0x5610fe864250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51606/0 (socket says 172.21.15.3:51606) 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f958b400640 1 -- 172.21.15.3:0/784561014 learned_addr learned my addr 172.21.15.3:0/784561014 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f958be00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95700a2db0 0x7f95700a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.574+0000 7f958aa00640 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe864790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.575+0000 7f958b400640 1 -- 172.21.15.3:0/784561014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe8673d0 msgr2=0x5610fe864cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.575+0000 7f958b400640 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe8673d0 0x5610fe864cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.575+0000 7f958b400640 1 -- 172.21.15.3:0/784561014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 msgr2=0x5610fe864790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.575+0000 7f958b400640 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe864790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.575+0000 7f958b400640 1 -- 172.21.15.3:0/784561014 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fe59b960 con 0x5610fe85c1f0 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.575+0000 7f958be00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95700a2db0 0x7f95700a5270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f958400a010 tx=0x7f9584004560 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.576+0000 7f958b400640 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 0x5610fe864250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f957c009640 tx=0x7f957c09ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a762ae7261cd7bd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.576+0000 7f9574c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.576+0000 7f9574c00640 1 -- 172.21.15.3:0/784561014 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0b4020 con 0x5610fe85c1f0 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.577+0000 7f9574c00640 1 -- 172.21.15.3:0/784561014 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f957c0bad70 con 0x5610fe85c1f0 2024-08-01T04:17:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.577+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9584003e50 con 0x5610fe85c1f0 2024-08-01T04:17:40.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.577+0000 7f9574c00640 1 -- 172.21.15.3:0/784561014 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f957c09bf30 con 0x5610fe85c1f0 2024-08-01T04:17:40.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.577+0000 7f9594ac2740 10 client.25577 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:40.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.577+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610fe875f70 con 0x5610fe85c1f0 2024-08-01T04:17:40.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.577+0000 7f9574c00640 1 -- 172.21.15.3:0/784561014 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9584003e50 con 0x5610fe85c1f0 2024-08-01T04:17:40.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9574c00640 1 -- 172.21.15.3:0/784561014 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f957c0cd470 con 0x5610fe85c1f0 2024-08-01T04:17:40.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9594ac2740 10 client.25577 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9594ac2740 10 client.25577 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9594ac2740 20 client.25577 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9594ac2740 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe877380 0x5610fe879770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9594ac2740 4 client.25577 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.578+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610fe7d5850 con 0x5610fe877380 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.579+0000 7f958aa00640 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe877380 0x5610fe879770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.579+0000 7f958aa00640 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe877380 0x5610fe879770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd1354c60c355be server_cookie=1dbd7f9f91162469 in_seq=0 out_seq=0 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.579+0000 7f9574c00640 10 client.25577 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.580+0000 7f9574c00640 1 -- 172.21.15.3:0/784561014 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5610fe7d5850 con 0x5610fe877380 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.580+0000 7f9574c00640 10 client.25577 handle_command_reply: tid=0 2024-08-01T04:17:40.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.580+0000 7f9594ac2740 1 client.25577 shutdown 2024-08-01T04:17:40.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.580+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 msgr2=0x5610fe864250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.580+0000 7f9594ac2740 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 0x5610fe864250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f957c009640 tx=0x7f957c09ba70 comp rx=0 tx=0).stop 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 shutdown_connections 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe877380 0x5610fe879770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe8673d0 0x5610fe864cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe864790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 --2- 172.21.15.3:0/784561014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85c1f0 0x5610fe864250 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 >> 172.21.15.3:0/784561014 conn(0x5610fe858780 msgr2=0x5610fe85e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9574c00640 0 client.25577 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 shutdown_connections 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 1 -- 172.21.15.3:0/784561014 wait complete. 2024-08-01T04:17:40.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.581+0000 7f9594ac2740 20 client.25577 trim_cache size 0 max 16384 2024-08-01T04:17:40.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 Processor -- start 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 -- start start 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe864b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe8577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe8673d0 0x5610fe87efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f958c1b65e0 con 0x5610fe85cb10 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f958c1b3eb0 con 0x5610fe8673d0 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.600+0000 7f9594ac2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f958c1b1d00 con 0x5610fe85c1f0 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe864b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe864b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55108/0 (socket says 172.21.15.3:55108) 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 -- 172.21.15.3:0/497142563 learned_addr learned my addr 172.21.15.3:0/497142563 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:40.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958be00640 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe8673d0 0x5610fe87efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958aa00640 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe8577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 -- 172.21.15.3:0/497142563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe8673d0 msgr2=0x5610fe87efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe8673d0 0x5610fe87efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 -- 172.21.15.3:0/497142563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 msgr2=0x5610fe8577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe8577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958b400640 1 -- 172.21.15.3:0/497142563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fe879d50 con 0x5610fe85c1f0 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.601+0000 7f958be00640 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe8673d0 0x5610fe87efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.602+0000 7f958b400640 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe864b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f957c098390 tx=0x7f957c01ffe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d748bbb2733c3c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.602+0000 7f9577400640 1 -- 172.21.15.3:0/497142563 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0b4070 con 0x5610fe85c1f0 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.602+0000 7f9577400640 1 -- 172.21.15.3:0/497142563 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f957c0a4ba0 con 0x5610fe85c1f0 2024-08-01T04:17:40.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.602+0000 7f9577400640 1 -- 172.21.15.3:0/497142563 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c09b5e0 con 0x5610fe85c1f0 2024-08-01T04:17:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 -- 172.21.15.3:0/497142563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 msgr2=0x5610fe864b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe864b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f957c098390 tx=0x7f957c01ffe0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 -- 172.21.15.3:0/497142563 shutdown_connections 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe8673d0 0x5610fe87efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe85cb10 0x5610fe8577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 --2- 172.21.15.3:0/497142563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe864b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.603+0000 7f9594ac2740 1 -- 172.21.15.3:0/497142563 >> 172.21.15.3:0/497142563 conn(0x5610fe858780 msgr2=0x5610fe8570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.604+0000 7f9594ac2740 1 -- 172.21.15.3:0/497142563 shutdown_connections 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.604+0000 7f9594ac2740 1 -- 172.21.15.3:0/497142563 wait complete. 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.604+0000 7f9594ac2740 1 -- start start 2024-08-01T04:17:40.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 Processor -- start 2024-08-01T04:17:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe86f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe860ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe861430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f958c1b2370 con 0x5610fe8673d0 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f958c1b37e0 con 0x5610fe85cb10 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f958c1b2a40 con 0x5610fe85c1f0 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.605+0000 7f9594ac2740 10 client.0 _ll_register_callbacks cb 0x7f957c0bc050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe86f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe860ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe86f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:55112/0 (socket says 172.21.15.3:55112) 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 -- 172.21.15.3:0/4158443997 learned_addr learned my addr 172.21.15.3:0/4158443997 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958be00640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe861430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 -- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 msgr2=0x5610fe860ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe860ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 -- 172.21.15.3:0/4158443997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 msgr2=0x5610fe861430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe861430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958b400640 1 -- 172.21.15.3:0/4158443997 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fe861b70 con 0x5610fe85c1f0 2024-08-01T04:17:40.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.606+0000 7f958aa00640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe860ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.607+0000 7f958b400640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe86f9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f957c09bc30 tx=0x7f957c00cfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c520fab8dd115b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:40.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.607+0000 7f9574c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.607+0000 7f9574c00640 1 -- 172.21.15.3:0/4158443997 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0b4070 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.607+0000 7f9574c00640 1 -- 172.21.15.3:0/4158443997 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f957c0cdaf0 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.607+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f957c0a4ba0 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.607+0000 7f9574c00640 1 -- 172.21.15.3:0/4158443997 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f957c09b890 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 10 client.25573 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610fe8e1ee0 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9574c00640 1 -- 172.21.15.3:0/4158443997 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f957c0a4ba0 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9574c00640 1 -- 172.21.15.3:0/4158443997 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f957c096c60 con 0x5610fe85c1f0 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 10 client.25573 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:40.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 10 client.25573 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 20 client.25573 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe8e23b0 0x5610fe8e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 4 client.25573 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5c87d78e", "release": true, "await": true}] 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.608+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5c87d78e", "release": true, "await": true}) -- 0x5610fe7cf3a0 con 0x5610fe8e23b0 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.609+0000 7f958aa00640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe8e23b0 0x5610fe8e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.610+0000 7f958aa00640 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe8e23b0 0x5610fe8e47a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee4c4f0beea0b12 server_cookie=c99551d42e0c828b in_seq=0 out_seq=0 2024-08-01T04:17:40.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.610+0000 7f9574c00640 10 client.25573 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.611+0000 7f9574c00640 1 -- 172.21.15.3:0/4158443997 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5610fe7cf3a0 con 0x5610fe8e23b0 2024-08-01T04:17:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.611+0000 7f9574c00640 10 client.25573 handle_command_reply: tid=0 2024-08-01T04:17:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 client.25573 shutdown 2024-08-01T04:17:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 msgr2=0x5610fe86f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe86f9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f957c09bc30 tx=0x7f957c00cfd0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 shutdown_connections 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5610fe8e23b0 0x5610fe8e47a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5610fe8673d0 0x5610fe861430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5610fe85cb10 0x5610fe860ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 --2- 172.21.15.3:0/4158443997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5610fe85c1f0 0x5610fe86f9a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9574c00640 0 client.25573 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.612+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 >> 172.21.15.3:0/4158443997 conn(0x5610fe858780 msgr2=0x5610fe8578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:40.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.613+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 shutdown_connections 2024-08-01T04:17:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.613+0000 7f9594ac2740 1 -- 172.21.15.3:0/4158443997 wait complete. 2024-08-01T04:17:40.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.613+0000 7f9594ac2740 20 client.25573 trim_cache size 0 max 16384 2024-08-01T04:17:40.616 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:40.616 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-08-01T04:17:40.616 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:17:40.616 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 4, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "5c87d78e": { 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 4, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.4, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:17:40.617 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:40.618 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.614+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95700a2db0 msgr2=0x7f95700a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.614+0000 7f9590a00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95700a2db0 0x7f95700a5270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f958400a010 tx=0x7f9584004560 comp rx=0 tx=0).stop 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.614+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 msgr2=0x7f958c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.614+0000 7f9590a00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 0x7f958c10d9a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f957c00e9f0 tx=0x7f957c00edf0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.615+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 shutdown_connections 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.615+0000 7f9590a00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f958c10dfb0 0x7f958c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.615+0000 7f9590a00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95700a2db0 0x7f95700a5270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.615+0000 7f9590a00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f958c074b60 0x7f958c10aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.615+0000 7f9590a00640 1 --2- 172.21.15.3:0/1710610617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f958c074240 0x7f958c10d9a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.615+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 >> 172.21.15.3:0/1710610617 conn(0x7f958c0fd470 msgr2=0x7f958c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:40.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.616+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 shutdown_connections 2024-08-01T04:17:40.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:40.617+0000 7f9590a00640 1 -- 172.21.15.3:0/1710610617 wait complete. 2024-08-01T04:17:41.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:41 smithi003 ceph-mon[32262]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:41 smithi160 ceph-mon[37224]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:41.537 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5c87d78e', total seconds elapsed: 29.0 2024-08-01T04:17:41.537 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.4 2024-08-01T04:17:41.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:17:41.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:41 smithi163 ceph-mon[37383]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-01T04:17:42.000 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.693+0000 7f8dad200640 1 -- 172.21.15.3:0/1698671487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074240 msgr2=0x7f8da8074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.693+0000 7f8dad200640 1 --2- 172.21.15.3:0/1698671487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074240 0x7f8da8074620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009900 tx=0x7f8d9802a480 comp rx=0 tx=0).stop 2024-08-01T04:17:42.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 -- 172.21.15.3:0/1698671487 shutdown_connections 2024-08-01T04:17:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 --2- 172.21.15.3:0/1698671487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da81103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 --2- 172.21.15.3:0/1698671487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8da8074b60 0x7f8da810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 --2- 172.21.15.3:0/1698671487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074240 0x7f8da8074620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 -- 172.21.15.3:0/1698671487 >> 172.21.15.3:0/1698671487 conn(0x7f8da80fd470 msgr2=0x7f8da80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 -- 172.21.15.3:0/1698671487 shutdown_connections 2024-08-01T04:17:42.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.694+0000 7f8dad200640 1 -- 172.21.15.3:0/1698671487 wait complete. 2024-08-01T04:17:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.695+0000 7f8dad200640 1 Processor -- start 2024-08-01T04:17:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.695+0000 7f8dad200640 1 -- start start 2024-08-01T04:17:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8dad200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8da8074240 0x7f8da819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8dad200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074b60 0x7f8da819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:42.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8dad200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8dad200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8113d30 con 0x7f8da8074b60 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8dad200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8113bb0 con 0x7f8da810dfb0 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8dad200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8113eb0 con 0x7f8da8074240 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.696+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da819eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60458/0 (socket says 172.21.15.3:60458) 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da7e00640 1 -- 172.21.15.3:0/1416644908 learned_addr learned my addr 172.21.15.3:0/1416644908 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da6a00640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074b60 0x7f8da819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:42.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da7400640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8da8074240 0x7f8da819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da6a00640 1 -- 172.21.15.3:0/1416644908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8da8074240 msgr2=0x7f8da819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da6a00640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8da8074240 0x7f8da819a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da6a00640 1 -- 172.21.15.3:0/1416644908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 msgr2=0x7f8da819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da6a00640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da6a00640 1 -- 172.21.15.3:0/1416644908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da819f180 con 0x7f8da8074b60 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.697+0000 7f8da7e00640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.698+0000 7f8da6a00640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074b60 0x7f8da819a710 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e9f0 tx=0x7f8d9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.698+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400ccc0 con 0x7f8da8074b60 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.698+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d94010910 con 0x7f8da8074b60 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.698+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94018960 con 0x7f8da8074b60 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.698+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da819f470 con 0x7f8da8074b60 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.698+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da81a6d50 con 0x7f8da8074b60 2024-08-01T04:17:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.700+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da8074620 con 0x7f8da8074b60 2024-08-01T04:17:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.704+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f8d94010ac0 con 0x7f8da8074b60 2024-08-01T04:17:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.706+0000 7f8d9fe00640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8d8c0a2f20 0x7f8d8c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.706+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8d940cff20 con 0x7f8da8074b60 2024-08-01T04:17:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.707+0000 7f8da7400640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8d8c0a2f20 0x7f8d8c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.708+0000 7f8da7400640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8d8c0a2f20 0x7f8d8c0a53e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8d98000c00 tx=0x7f8d98005b50 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:42.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.710+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8d9409c270 con 0x7f8da8074b60 2024-08-01T04:17:42.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.938+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8da819b530 con 0x7f8da8074b60 2024-08-01T04:17:42.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.939+0000 7f8d9fe00640 1 -- 172.21.15.3:0/1416644908 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7f8d940a0570 con 0x7f8da8074b60 2024-08-01T04:17:42.941 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:17:42.942 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:17:42.942 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:17:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.942+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8d8c0a2f20 msgr2=0x7f8d8c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.942+0000 7f8dad200640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8d8c0a2f20 0x7f8d8c0a53e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8d98000c00 tx=0x7f8d98005b50 comp rx=0 tx=0).stop 2024-08-01T04:17:42.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.942+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074b60 msgr2=0x7f8da819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.942+0000 7f8dad200640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074b60 0x7f8da819a710 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e9f0 tx=0x7f8d9400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 shutdown_connections 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da810dfb0 0x7f8da819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8da8074b60 0x7f8da819a710 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f8d8c0a2f20 0x7f8d8c0a53e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 --2- 172.21.15.3:0/1416644908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8da8074240 0x7f8da819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 >> 172.21.15.3:0/1416644908 conn(0x7f8da80fd470 msgr2=0x7f8da80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:42.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 shutdown_connections 2024-08-01T04:17:42.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:42.943+0000 7f8dad200640 1 -- 172.21.15.3:0/1416644908 wait complete. 2024-08-01T04:17:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:43 smithi003 ceph-mon[32262]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-01T04:17:43.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:43 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1416644908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:43 smithi160 ceph-mon[37224]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-01T04:17:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:43 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1416644908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:43.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:43 smithi163 ceph-mon[37383]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-01T04:17:43.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:43 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1416644908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:17:43.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db / --timeout 90 --expiration 144.4 --await-for 45.0' 2024-08-01T04:17:43.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db / --timeout 90 --expiration 144.4 --await-for 45.0 2024-08-01T04:17:44.428 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 -- 172.21.15.3:0/3184981182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 msgr2=0x7f97d8102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 --2- 172.21.15.3:0/3184981182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 0x7f97d8102ca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009900 tx=0x7f97c802a480 comp rx=0 tx=0).stop 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 -- 172.21.15.3:0/3184981182 shutdown_connections 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 --2- 172.21.15.3:0/3184981182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d810bcf0 0x7f97d810e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 --2- 172.21.15.3:0/3184981182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d81031e0 0x7f97d810b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 --2- 172.21.15.3:0/3184981182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 0x7f97d8102ca0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 -- 172.21.15.3:0/3184981182 >> 172.21.15.3:0/3184981182 conn(0x7f97d80fc340 msgr2=0x7f97d80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:45.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 -- 172.21.15.3:0/3184981182 shutdown_connections 2024-08-01T04:17:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.210+0000 7f97d7200640 1 -- 172.21.15.3:0/3184981182 wait complete. 2024-08-01T04:17:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.212+0000 7f97d7200640 1 Processor -- start 2024-08-01T04:17:45.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.212+0000 7f97d7200640 1 -- start start 2024-08-01T04:17:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.212+0000 7f97d7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 0x7f97d8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d7200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 0x7f97d81004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d810bcf0 0x7f97d8100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8111b40 con 0x7f97d81028c0 2024-08-01T04:17:45.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d7200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81119c0 con 0x7f97d81031e0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8111cc0 con 0x7f97d810bcf0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 0x7f97d81004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d810bcf0 0x7f97d8100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 0x7f97d81004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60466/0 (socket says 172.21.15.3:60466) 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d5400640 1 -- 172.21.15.3:0/4028997001 learned_addr learned my addr 172.21.15.3:0/4028997001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d5e00640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 0x7f97d8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d6800640 1 -- 172.21.15.3:0/4028997001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 msgr2=0x7f97d81004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d6800640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 0x7f97d81004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d6800640 1 -- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 msgr2=0x7f97d8101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d6800640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 0x7f97d8101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d6800640 1 -- 172.21.15.3:0/4028997001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d81010d0 con 0x7f97d810bcf0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.213+0000 7f97d5400640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 0x7f97d81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.214+0000 7f97d6800640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d810bcf0 0x7f97d8100a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009fd0 tx=0x7f97d000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:45.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.214+0000 7f97cea00640 1 -- 172.21.15.3:0/4028997001 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d000ce20 con 0x7f97d810bcf0 2024-08-01T04:17:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.214+0000 7f97cea00640 1 -- 172.21.15.3:0/4028997001 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0005020 con 0x7f97d810bcf0 2024-08-01T04:17:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.214+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d81af040 con 0x7f97d810bcf0 2024-08-01T04:17:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.214+0000 7f97cea00640 1 -- 172.21.15.3:0/4028997001 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0010720 con 0x7f97d810bcf0 2024-08-01T04:17:45.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.214+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d81af3e0 con 0x7f97d810bcf0 2024-08-01T04:17:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.219+0000 7f97dfa67740 1 Processor -- start 2024-08-01T04:17:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.219+0000 7f97cea00640 1 -- 172.21.15.3:0/4028997001 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 25) ==== 138116+0+0 (secure 0 0 0) 0x7f97d0005540 con 0x7f97d810bcf0 2024-08-01T04:17:45.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.219+0000 7f97dfa67740 1 -- start start 2024-08-01T04:17:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.220+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca95f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.220+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3570 0x563ca95f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.220+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca9608f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.220+0000 7f97dfa67740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563ca9558b90 con 0x563ca95f3570 2024-08-01T04:17:45.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.220+0000 7f97dfa67740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563ca94f01d0 con 0x563ca9606b20 2024-08-01T04:17:45.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.220+0000 7f97dfa67740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563ca9554b50 con 0x563ca95f4ae0 2024-08-01T04:17:45.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.221+0000 7f97d5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3570 0x563ca95f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.221+0000 7f97d5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca95f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.221+0000 7f97d5400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca95f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:35384/0 (socket says 172.21.15.3:35384) 2024-08-01T04:17:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.221+0000 7f97d5400640 1 -- 172.21.15.3:0/667248695 learned_addr learned my addr 172.21.15.3:0/667248695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.221+0000 7f97d5400640 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca9608f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.221+0000 7f97cea00640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f97bc0a2f20 0x7f97bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97cea00640 1 -- 172.21.15.3:0/4028997001 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f97d00cfeb0 con 0x7f97d810bcf0 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97d6800640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f97bc0a2f20 0x7f97bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97d5e00640 1 -- 172.21.15.3:0/667248695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 msgr2=0x563ca95f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97d5e00640 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca95f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97d5e00640 1 -- 172.21.15.3:0/667248695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 msgr2=0x563ca9608f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97d5e00640 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca9608f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.222+0000 7f97d5e00640 1 -- 172.21.15.3:0/667248695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca92cab90 con 0x563ca95f3570 2024-08-01T04:17:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.223+0000 7f97d5e00640 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3570 0x563ca95f39b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97c802aa70 tx=0x7f97c8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75059011ac90d1b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.223+0000 7f97d6800640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f97bc0a2f20 0x7f97bc0a53e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97d00cd3e0 tx=0x7f97d0005f70 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.223+0000 7f97c3400640 1 -- 172.21.15.3:0/667248695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8035030 con 0x563ca95f3570 2024-08-01T04:17:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.224+0000 7f97c3400640 1 -- 172.21.15.3:0/667248695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c80043c0 con 0x563ca95f3570 2024-08-01T04:17:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.224+0000 7f97c3400640 1 -- 172.21.15.3:0/667248695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8004dd0 con 0x563ca95f3570 2024-08-01T04:17:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.224+0000 7f97dfa67740 1 -- 172.21.15.3:0/667248695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3570 msgr2=0x563ca95f39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.224+0000 7f97dfa67740 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3570 0x563ca95f39b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97c802aa70 tx=0x7f97c8005e60 comp rx=0 tx=0).stop 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 -- 172.21.15.3:0/667248695 shutdown_connections 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca9608f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3570 0x563ca95f39b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 --2- 172.21.15.3:0/667248695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca95f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 -- 172.21.15.3:0/667248695 >> 172.21.15.3:0/667248695 conn(0x563ca95bc330 msgr2=0x563ca95efc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 -- 172.21.15.3:0/667248695 shutdown_connections 2024-08-01T04:17:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.225+0000 7f97dfa67740 1 -- 172.21.15.3:0/667248695 wait complete. 2024-08-01T04:17:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.226+0000 7f97dfa67740 1 -- start start 2024-08-01T04:17:45.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.226+0000 7f97dfa67740 1 Processor -- start 2024-08-01T04:17:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f3570 0x563ca9603b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca9604080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca96045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563ca96097c0 con 0x563ca95f4ae0 2024-08-01T04:17:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x563ca9513920 con 0x563ca9606b20 2024-08-01T04:17:45.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x563ca9609940 con 0x563ca95f3570 2024-08-01T04:17:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97dfa67740 10 client.0 _ll_register_callbacks cb 0x563ca9268090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97d5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca9604080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97d5e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca9604080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53092/0 (socket says 172.21.15.3:53092) 2024-08-01T04:17:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97d5e00640 1 -- 172.21.15.3:0/841091704 learned_addr learned my addr 172.21.15.3:0/841091704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.227+0000 7f97d5400640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f3570 0x563ca9603b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5400640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca96045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5e00640 1 -- 172.21.15.3:0/841091704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f3570 msgr2=0x563ca9603b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5e00640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f3570 0x563ca9603b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5e00640 1 -- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 msgr2=0x563ca96045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5e00640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca96045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5e00640 1 -- 172.21.15.3:0/841091704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca9612e70 con 0x563ca95f4ae0 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.228+0000 7f97d5400640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f3570 0x563ca9603b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.229+0000 7f97d5e00640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca9604080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97c8000c00 tx=0x7f97c8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59627af8bb665943 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:45.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.229+0000 7f97b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:17:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.229+0000 7f97b7e00640 1 -- 172.21.15.3:0/841091704 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8035070 con 0x563ca95f4ae0 2024-08-01T04:17:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.229+0000 7f97b7e00640 1 -- 172.21.15.3:0/841091704 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f97c80317c0 con 0x563ca95f4ae0 2024-08-01T04:17:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.229+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c80043c0 con 0x563ca95f4ae0 2024-08-01T04:17:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.230+0000 7f97b7e00640 1 -- 172.21.15.3:0/841091704 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97c8009520 con 0x563ca95f4ae0 2024-08-01T04:17:45.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.230+0000 7f97dfa67740 10 client.15456 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:45.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.230+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ca96157a0 con 0x563ca95f4ae0 2024-08-01T04:17:45.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.230+0000 7f97b7e00640 1 -- 172.21.15.3:0/841091704 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c80043c0 con 0x563ca95f4ae0 2024-08-01T04:17:45.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.230+0000 7f97b7e00640 1 -- 172.21.15.3:0/841091704 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f97c803b640 con 0x563ca95f4ae0 2024-08-01T04:17:45.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97dfa67740 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97dfa67740 10 client.15456 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97dfa67740 20 client.15456 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97dfa67740 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9616af0 0x563ca9618ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97dfa67740 4 client.15456 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca952d320 con 0x563ca9616af0 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.231+0000 7f97d5400640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9616af0 0x563ca9618ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.232+0000 7f97d5400640 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9616af0 0x563ca9618ee0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a53e6ae6b5d74b server_cookie=ed006dff3a6d0df3 in_seq=0 out_seq=0 2024-08-01T04:17:45.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.232+0000 7f97b7e00640 10 client.15456 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97b7e00640 1 -- 172.21.15.3:0/841091704 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ca952d320 con 0x563ca9616af0 2024-08-01T04:17:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97b7e00640 10 client.15456 handle_command_reply: tid=0 2024-08-01T04:17:45.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 client.15456 shutdown 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 msgr2=0x563ca9604080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca9604080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97c8000c00 tx=0x7f97c8003cd0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 shutdown_connections 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9616af0 0x563ca9618ee0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97b7e00640 0 client.15456 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca9606b20 0x563ca96045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca9604080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 --2- 172.21.15.3:0/841091704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f3570 0x563ca9603b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.233+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 >> 172.21.15.3:0/841091704 conn(0x563ca95bc330 msgr2=0x563ca95f5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:45.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.234+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 shutdown_connections 2024-08-01T04:17:45.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.234+0000 7f97dfa67740 1 -- 172.21.15.3:0/841091704 wait complete. 2024-08-01T04:17:45.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.234+0000 7f97dfa67740 20 client.15456 trim_cache size 0 max 16384 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.255+0000 7f97dfa67740 1 Processor -- start 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.255+0000 7f97dfa67740 1 -- start start 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9605ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca961bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca9606b20 0x563ca961c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97dfa67740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b6360 con 0x563ca9606b20 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97dfa67740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b3c30 con 0x563ca95f3570 2024-08-01T04:17:45.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97dfa67740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b1a80 con 0x563ca95f4ae0 2024-08-01T04:17:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97d5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca961bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97d5e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca961bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:35394/0 (socket says 172.21.15.3:35394) 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.256+0000 7f97d5e00640 1 -- 172.21.15.3:0/792042554 learned_addr learned my addr 172.21.15.3:0/792042554 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9605ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca9606b20 0x563ca961c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 -- 172.21.15.3:0/792042554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 msgr2=0x563ca961bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca961bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 -- 172.21.15.3:0/792042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca9606b20 msgr2=0x563ca961c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca9606b20 0x563ca961c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.257+0000 7f97d5400640 1 -- 172.21.15.3:0/792042554 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca95f60c0 con 0x563ca95f3570 2024-08-01T04:17:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.258+0000 7f97d5400640 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9605ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x563ca94f02f0 tx=0x7f97c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05432de8e6151e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:45.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.258+0000 7f97c2a00640 1 -- 172.21.15.3:0/792042554 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4034070 con 0x563ca95f3570 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.259+0000 7f97c2a00640 1 -- 172.21.15.3:0/792042554 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c4026be0 con 0x563ca95f3570 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.259+0000 7f97c2a00640 1 -- 172.21.15.3:0/792042554 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4015430 con 0x563ca95f3570 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.259+0000 7f97dfa67740 1 -- 172.21.15.3:0/792042554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 msgr2=0x563ca9605ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.259+0000 7f97dfa67740 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9605ab0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x563ca94f02f0 tx=0x7f97c4004730 comp rx=0 tx=0).stop 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 -- 172.21.15.3:0/792042554 shutdown_connections 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca9606b20 0x563ca961c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca95f4ae0 0x563ca961bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 --2- 172.21.15.3:0/792042554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9605ab0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 -- 172.21.15.3:0/792042554 >> 172.21.15.3:0/792042554 conn(0x563ca95bc330 msgr2=0x563ca96053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 -- 172.21.15.3:0/792042554 shutdown_connections 2024-08-01T04:17:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 -- 172.21.15.3:0/792042554 wait complete. 2024-08-01T04:17:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.260+0000 7f97dfa67740 1 -- start start 2024-08-01T04:17:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 Processor -- start 2024-08-01T04:17:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9619cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca961a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca9606b20 0x563ca961a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b20f0 con 0x563ca95f4ae0 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b3560 con 0x563ca95f3570 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b27c0 con 0x563ca9606b20 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97dfa67740 10 client.0 _ll_register_callbacks cb 0x7f97d806ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.261+0000 7f97d5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9619cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9619cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60514/0 (socket says 172.21.15.3:60514) 2024-08-01T04:17:45.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 -- 172.21.15.3:0/1350493404 learned_addr learned my addr 172.21.15.3:0/1350493404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca9606b20 0x563ca961a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5e00640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca961a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 -- 172.21.15.3:0/1350493404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca9606b20 msgr2=0x563ca961a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca9606b20 0x563ca961a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 -- 172.21.15.3:0/1350493404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 msgr2=0x563ca961a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca961a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 -- 172.21.15.3:0/1350493404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca9680480 con 0x563ca95f3570 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97d5400640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9619cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97c4026dd0 tx=0x7f97c402ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13077e6a55db5f75 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.262+0000 7f97b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:17:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.263+0000 7f97b7e00640 1 -- 172.21.15.3:0/1350493404 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4040060 con 0x563ca95f3570 2024-08-01T04:17:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.263+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97c4026be0 con 0x563ca95f3570 2024-08-01T04:17:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.263+0000 7f97b7e00640 1 -- 172.21.15.3:0/1350493404 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f97c402f840 con 0x563ca95f3570 2024-08-01T04:17:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.263+0000 7f97b7e00640 1 -- 172.21.15.3:0/1350493404 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97c4034040 con 0x563ca95f3570 2024-08-01T04:17:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.263+0000 7f97dfa67740 10 client.25607 fetch_fsmap learned FSMap version 26 2024-08-01T04:17:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.263+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563ca9681720 con 0x563ca95f3570 2024-08-01T04:17:45.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97b7e00640 1 -- 172.21.15.3:0/1350493404 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97c4026be0 con 0x563ca95f3570 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97b7e00640 1 -- 172.21.15.3:0/1350493404 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f97c402c070 con 0x563ca95f3570 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97dfa67740 10 client.25607 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97dfa67740 10 client.25607 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97dfa67740 20 client.25607 populate_metadata read hostname 'smithi003' 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97dfa67740 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9681cc0 0x563ca96840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97dfa67740 4 client.25607 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.4, "await_for": 45.0}] 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.264+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.4, "await_for": 45.0}) -- 0x563ca95662c0 con 0x563ca9681cc0 2024-08-01T04:17:45.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.265+0000 7f97d5e00640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9681cc0 0x563ca96840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:17:45.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.265+0000 7f97d5e00640 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9681cc0 0x563ca96840b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773df3d1abaeb343 server_cookie=779430f965e79786 in_seq=0 out_seq=0 2024-08-01T04:17:45.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.265+0000 7f97b7e00640 10 client.25607 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:45.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.274+0000 7f97b7e00640 1 -- 172.21.15.3:0/1350493404 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563ca95662c0 con 0x563ca9681cc0 2024-08-01T04:17:45.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.274+0000 7f97b7e00640 10 client.25607 handle_command_reply: tid=0 2024-08-01T04:17:45.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.274+0000 7f97dfa67740 1 client.25607 shutdown 2024-08-01T04:17:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 msgr2=0x563ca9619cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9619cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97c4026dd0 tx=0x7f97c402ff20 comp rx=0 tx=0).stop 2024-08-01T04:17:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 shutdown_connections 2024-08-01T04:17:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x563ca9681cc0 0x563ca96840b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563ca9606b20 0x563ca961a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97b7e00640 0 client.25607 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:17:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4ae0 0x563ca961a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 --2- 172.21.15.3:0/1350493404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563ca95f3570 0x563ca9619cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 >> 172.21.15.3:0/1350493404 conn(0x563ca95bc330 msgr2=0x563ca95f58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 shutdown_connections 2024-08-01T04:17:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.275+0000 7f97dfa67740 1 -- 172.21.15.3:0/1350493404 wait complete. 2024-08-01T04:17:45.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.276+0000 7f97dfa67740 20 client.25607 trim_cache size 0 max 16384 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 6, 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "2dc67b78": { 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 6, 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:17:45.279 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.4, 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:45.280 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:45.281 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:17:45.281 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:17:45.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f97bc0a2f20 msgr2=0x7f97bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f97bc0a2f20 0x7f97bc0a53e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f97d00cd3e0 tx=0x7f97d0005f70 comp rx=0 tx=0).stop 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d810bcf0 msgr2=0x7f97d8100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d810bcf0 0x7f97d8100a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009fd0 tx=0x7f97d000ef10 comp rx=0 tx=0).stop 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 shutdown_connections 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f97bc0a2f20 0x7f97bc0a53e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d810bcf0 0x7f97d8100a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f97d81031e0 0x7f97d81004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 --2- 172.21.15.3:0/4028997001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97d81028c0 0x7f97d8101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:17:45.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.277+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 >> 172.21.15.3:0/4028997001 conn(0x7f97d80fc340 msgr2=0x7f97d80fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:17:45.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.278+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 shutdown_connections 2024-08-01T04:17:45.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:17:45.278+0000 7f97d7200640 1 -- 172.21.15.3:0/4028997001 wait complete. 2024-08-01T04:17:45.347 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:45 smithi163 ceph-mon[37383]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-01T04:17:45.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:45 smithi160 ceph-mon[37224]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-01T04:17:45.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:45 smithi003 ceph-mon[32262]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-01T04:17:46.207 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2dc67b78', quiesce took 4.7 seconds. Will release after: 19.7 2024-08-01T04:17:47.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:47 smithi003 ceph-mon[32262]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-01T04:17:47.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:47 smithi160 ceph-mon[37224]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-01T04:17:47.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:47 smithi163 ceph-mon[37383]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-01T04:17:49.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:49 smithi160 ceph-mon[37224]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:49.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:49 smithi160 ceph-mon[37224]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-08-01T04:17:49.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:49 smithi163 ceph-mon[37383]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:49.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:49 smithi163 ceph-mon[37383]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-08-01T04:17:49.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:49 smithi003 ceph-mon[32262]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:49.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:49 smithi003 ceph-mon[32262]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-08-01T04:17:51.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:51 smithi160 ceph-mon[37224]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:51.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:51 smithi163 ceph-mon[37383]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:51.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:51 smithi003 ceph-mon[32262]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:53.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:53 smithi160 ceph-mon[37224]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:53.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:53 smithi163 ceph-mon[37383]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:53.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:53 smithi003 ceph-mon[32262]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:55.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:55 smithi160 ceph-mon[37224]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:55.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:55 smithi163 ceph-mon[37383]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:55.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:55 smithi003 ceph-mon[32262]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:57.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:57 smithi160 ceph-mon[37224]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:57.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:57 smithi163 ceph-mon[37383]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:57.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:57 smithi003 ceph-mon[32262]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-08-01T04:17:57.548 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 60% (7774/12824) Updating files: 61% (7823/12824) Updating files: 62% (7951/12824) Updating files: 63% (8080/12824) Updating files: 64% (8208/12824) Updating files: 65% (8336/12824) Updating files: 66% (8464/12824) Updating files: 67% (8593/12824) Updating files: 68% (8721/12824) Updating files: 69% (8849/12824) Updating files: 70% (8977/12824) Updating files: 71% (9106/12824) Updating files: 72% (9234/12824) Updating files: 73% (9362/12824) Updating files: 74% (9490/12824) Updating files: 75% (9618/12824) Updating files: 76% (9747/12824) Updating files: 77% (9875/12824) Updating files: 78% (10003/12824) Updating files: 79% (10131/12824) Updating files: 80% (10260/12824) Updating files: 81% (10388/12824) Updating files: 82% (10516/12824) Updating files: 83% (10644/12824) Updating files: 84% (10773/12824) Updating files: 85% (10901/12824) Updating files: 86% (11029/12824) Updating files: 87% (11157/12824) Updating files: 88% (11286/12824) Updating files: 89% (11414/12824) Updating files: 90% (11542/12824) Updating files: 91% (11670/12824) Updating files: 92% (11799/12824) Updating files: 93% (11927/12824) Updating files: 94% (12055/12824) Updating files: 95% (12183/12824) Updating files: 96% (12312/12824) Updating files: 97% (12440/12824) Updating files: 97% (12513/12824) Updating files: 98% (12568/12824) Updating files: 99% (12696/12824) Updating files: 100% (12824/12824) Updating files: 100% (12824/12824), done. 2024-08-01T04:17:58.373 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to 'd99c9769be1521395979e8c23d9073adbf1ff43f'. 2024-08-01T04:17:58.373 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.373 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-01T04:17:58.373 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-01T04:17:58.373 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.374 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-08-01T04:17:58.375 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.375 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-08-01T04:17:58.375 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.375 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-01T04:17:58.375 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-08-01T04:17:58.375 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at d99c9769be1 Merge pull request #58890 from Matan-B/wip-matanb-logentry-string-1 2024-08-01T04:17:58.386 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-01T04:17:58.445 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-01T04:17:58.453 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-01T04:17:58.453 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-01T04:17:59.279 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-01T04:17:59.390 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-01T04:17:59.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:17:59 smithi160 ceph-mon[37224]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:59.480 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-01T04:17:59.486 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-01T04:17:59.486 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-01T04:17:59.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:17:59 smithi163 ceph-mon[37383]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:59.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:17:59 smithi003 ceph-mon[32262]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-01T04:17:59.577 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-01T04:17:59.588 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:17:59.588 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-01T04:17:59.640 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-08-01T04:17:59.641 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-08-01T04:17:59.641 DEBUG:teuthology.orchestra.run.smithi003:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d99c9769be1521395979e8c23d9073adbf1ff43f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-08-01T04:18:01.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:01 smithi160 ceph-mon[37224]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:18:01.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:01 smithi163 ceph-mon[37383]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:18:01.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:01 smithi003 ceph-mon[32262]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:18:03.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:03 smithi160 ceph-mon[37224]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-01T04:18:03.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:03 smithi163 ceph-mon[37383]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-01T04:18:03.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:03 smithi003 ceph-mon[32262]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-01T04:18:04.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:18:05.113 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:05.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:05 smithi160 ceph-mon[37224]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:05.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:05 smithi003 ceph-mon[32262]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:05.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:05 smithi163 ceph-mon[37383]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:05.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 -- 172.21.15.3:0/406540842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 msgr2=0x7fab4c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 --2- 172.21.15.3:0/406540842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c10e790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009930 tx=0x7fab3c02a420 comp rx=0 tx=0).stop 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 -- 172.21.15.3:0/406540842 shutdown_connections 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 --2- 172.21.15.3:0/406540842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c10e790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 --2- 172.21.15.3:0/406540842 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c1018c0 0x7fab4c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 --2- 172.21.15.3:0/406540842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1078c0 0x7fab4c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 -- 172.21.15.3:0/406540842 >> 172.21.15.3:0/406540842 conn(0x7fab4c0fd620 msgr2=0x7fab4c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 -- 172.21.15.3:0/406540842 shutdown_connections 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.871+0000 7fab52800640 1 -- 172.21.15.3:0/406540842 wait complete. 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.872+0000 7fab52800640 1 Processor -- start 2024-08-01T04:18:05.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 -- start start 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1018c0 0x7fab4c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c1078c0 0x7fab4c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c111d10 con 0x7fab4c1018c0 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c111b90 con 0x7fab4c1078c0 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.873+0000 7fab52800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c111e90 con 0x7fab4c102260 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1018c0 0x7fab4c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1018c0 0x7fab4c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55308/0 (socket says 172.21.15.3:55308) 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4b400640 1 -- 172.21.15.3:0/2323536317 learned_addr learned my addr 172.21.15.3:0/2323536317 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4be00640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c1078c0 0x7fab4c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4aa00640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4aa00640 1 -- 172.21.15.3:0/2323536317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c1078c0 msgr2=0x7fab4c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4aa00640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c1078c0 0x7fab4c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4aa00640 1 -- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1018c0 msgr2=0x7fab4c19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4aa00640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1018c0 0x7fab4c19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.874+0000 7fab4aa00640 1 -- 172.21.15.3:0/2323536317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c19f350 con 0x7fab4c102260 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.875+0000 7fab4aa00640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c19a8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab3400e9f0 tx=0x7fab3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.875+0000 7fab43e00640 1 -- 172.21.15.3:0/2323536317 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3400ccc0 con 0x7fab4c102260 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.875+0000 7fab43e00640 1 -- 172.21.15.3:0/2323536317 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab34005020 con 0x7fab4c102260 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.875+0000 7fab43e00640 1 -- 172.21.15.3:0/2323536317 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab340049e0 con 0x7fab4c102260 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.875+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c19f640 con 0x7fab4c102260 2024-08-01T04:18:05.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.875+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1a6f20 con 0x7fab4c102260 2024-08-01T04:18:05.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.879+0000 7fab43e00640 1 -- 172.21.15.3:0/2323536317 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fab34005560 con 0x7fab4c102260 2024-08-01T04:18:05.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.879+0000 7fab54ffd740 1 Processor -- start 2024-08-01T04:18:05.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.879+0000 7fab54ffd740 1 -- start start 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 0x556c98d79f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d84830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d87160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab54ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556c98cdfbf0 con 0x556c98d84d70 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab54ffd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556c98c77bd0 con 0x556c98d7a4b0 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab54ffd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556c98cdbbb0 con 0x556c98d79b90 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 0x556c98d79f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab4aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d87160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 0x556c98d79f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:52898/0 (socket says 172.21.15.3:52898) 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.880+0000 7fab4be00640 1 -- 172.21.15.3:0/1759557184 learned_addr learned my addr 172.21.15.3:0/1759557184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.881+0000 7fab4b400640 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d84830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.881+0000 7fab4be00640 1 -- 172.21.15.3:0/1759557184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 msgr2=0x556c98d84830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.881+0000 7fab4be00640 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d84830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.881+0000 7fab4be00640 1 -- 172.21.15.3:0/1759557184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 msgr2=0x556c98d87160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.881+0000 7fab4be00640 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d87160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.881+0000 7fab4be00640 1 -- 172.21.15.3:0/1759557184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c98c7e8a0 con 0x556c98d79b90 2024-08-01T04:18:05.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab4be00640 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 0x556c98d79f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009930 tx=0x7fab3c005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886105e69212ff90 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab43e00640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab300a2d70 0x7fab300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab43e00640 1 -- 172.21.15.3:0/2323536317 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fab340cf400 con 0x7fab4c102260 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab40c00640 1 -- 172.21.15.3:0/1759557184 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c040070 con 0x556c98d79b90 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab40c00640 1 -- 172.21.15.3:0/1759557184 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3c004340 con 0x556c98d79b90 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab40c00640 1 -- 172.21.15.3:0/1759557184 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035040 con 0x556c98d79b90 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.882+0000 7fab4b400640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab300a2d70 0x7fab300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 -- 172.21.15.3:0/1759557184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 msgr2=0x556c98d79f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 0x556c98d79f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009930 tx=0x7fab3c005e20 comp rx=0 tx=0).stop 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab4b400640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab300a2d70 0x7fab300a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x556c98d76f80 tx=0x7fab44009800 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 -- 172.21.15.3:0/1759557184 shutdown_connections 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d87160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d84830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 --2- 172.21.15.3:0/1759557184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d79b90 0x556c98d79f70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 -- 172.21.15.3:0/1759557184 >> 172.21.15.3:0/1759557184 conn(0x556c98d76120 msgr2=0x556c98d76500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 -- 172.21.15.3:0/1759557184 shutdown_connections 2024-08-01T04:18:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.883+0000 7fab54ffd740 1 -- 172.21.15.3:0/1759557184 wait complete. 2024-08-01T04:18:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.884+0000 7fab54ffd740 1 -- start start 2024-08-01T04:18:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.884+0000 7fab54ffd740 1 Processor -- start 2024-08-01T04:18:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.884+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d81c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.884+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 0x556c98d82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab54ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556c98d87940 con 0x556c98d79b90 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab54ffd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556c98c9ad40 con 0x556c98d7a4b0 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab54ffd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556c98d87ac0 con 0x556c98d84d70 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab54ffd740 10 client.0 _ll_register_callbacks cb 0x556c98c63b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 0x556c98d82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d81c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 0x556c98d82700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:52900/0 (socket says 172.21.15.3:52900) 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d81c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55318/0 (socket says 172.21.15.3:55318) 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 -- 172.21.15.3:0/3490190970 learned_addr learned my addr 172.21.15.3:0/3490190970 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 -- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 msgr2=0x556c98d821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 -- 172.21.15.3:0/3490190970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 msgr2=0x556c98d81c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d81c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.885+0000 7fab4be00640 1 -- 172.21.15.3:0/3490190970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c98a513c0 con 0x556c98d84d70 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab4be00640 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 0x556c98d82700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab3c02aa10 tx=0x7fab3c003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42e69bb7588dc6c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab39600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab39600640 1 -- 172.21.15.3:0/3490190970 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035070 con 0x556c98d84d70 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab3c004340 con 0x556c98d84d70 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab39600640 1 -- 172.21.15.3:0/3490190970 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fab3c032d90 con 0x556c98d84d70 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab39600640 1 -- 172.21.15.3:0/3490190970 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab3c056050 con 0x556c98d84d70 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.886+0000 7fab54ffd740 10 client.25609 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:05.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c98d87990 con 0x556c98d84d70 2024-08-01T04:18:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab39600640 1 -- 172.21.15.3:0/3490190970 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab3c004340 con 0x556c98d84d70 2024-08-01T04:18:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab39600640 1 -- 172.21.15.3:0/3490190970 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fab3c004ab0 con 0x556c98d84d70 2024-08-01T04:18:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 10 client.25609 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 20 client.25609 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98d947a0 0x556c98d96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 4 client.25609 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.887+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c98ced160 con 0x556c98d947a0 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.888+0000 7fab4b400640 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98d947a0 0x556c98d96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.888+0000 7fab4b400640 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98d947a0 0x556c98d96b90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7edf40f4eaff95f server_cookie=7a1068f6ae7681da in_seq=0 out_seq=0 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.888+0000 7fab39600640 10 client.25609 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.889+0000 7fab39600640 1 -- 172.21.15.3:0/3490190970 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556c98ced160 con 0x556c98d947a0 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.889+0000 7fab39600640 10 client.25609 handle_command_reply: tid=0 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.889+0000 7fab54ffd740 1 client.25609 shutdown 2024-08-01T04:18:05.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.889+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 msgr2=0x556c98d82700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.889+0000 7fab54ffd740 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 0x556c98d82700 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fab3c02aa10 tx=0x7fab3c003f80 comp rx=0 tx=0).stop 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 shutdown_connections 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98d947a0 0x556c98d96b90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d84d70 0x556c98d82700 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d7a4b0 0x556c98d821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 --2- 172.21.15.3:0/3490190970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d81c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 >> 172.21.15.3:0/3490190970 conn(0x556c98d76120 msgr2=0x556c98d7be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab39600640 0 client.25609 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 shutdown_connections 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 1 -- 172.21.15.3:0/3490190970 wait complete. 2024-08-01T04:18:05.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.890+0000 7fab54ffd740 20 client.25609 trim_cache size 0 max 16384 2024-08-01T04:18:05.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:18:05.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 Processor -- start 2024-08-01T04:18:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 -- start start 2024-08-01T04:18:05.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 0x556c98d7c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d9c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d9c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1b2220 con 0x556c98d84d70 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1afaf0 con 0x556c98d79b90 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.910+0000 7fab54ffd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ad940 con 0x556c98d7a4b0 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 0x556c98d7c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d9c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 0x556c98d7c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:53154/0 (socket says 172.21.15.3:53154) 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d9c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55324/0 (socket says 172.21.15.3:55324) 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 -- 172.21.15.3:0/133712449 learned_addr learned my addr 172.21.15.3:0/133712449 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:05.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4aa00640 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 -- 172.21.15.3:0/133712449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 msgr2=0x556c98d9c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d9c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 -- 172.21.15.3:0/133712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 msgr2=0x556c98d9c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.911+0000 7fab4b400640 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d9c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.912+0000 7fab4b400640 1 -- 172.21.15.3:0/133712449 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c98d771d0 con 0x556c98d79b90 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.912+0000 7fab4be00640 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d9c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.912+0000 7fab4aa00640 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d9c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.912+0000 7fab4b400640 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 0x556c98d7c5f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab44009470 tx=0x7fab44019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d039dfa7ce39bcb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.913+0000 7fab3be00640 1 -- 172.21.15.3:0/133712449 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4402a4e0 con 0x556c98d79b90 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.913+0000 7fab3be00640 1 -- 172.21.15.3:0/133712449 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4402ab60 con 0x556c98d79b90 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.913+0000 7fab3be00640 1 -- 172.21.15.3:0/133712449 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4401d410 con 0x556c98d79b90 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.913+0000 7fab54ffd740 1 -- 172.21.15.3:0/133712449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 msgr2=0x556c98d7c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.913+0000 7fab54ffd740 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 0x556c98d7c5f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab44009470 tx=0x7fab44019000 comp rx=0 tx=0).stop 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 -- 172.21.15.3:0/133712449 shutdown_connections 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d84d70 0x556c98d9c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d9c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 --2- 172.21.15.3:0/133712449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d79b90 0x556c98d7c5f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 -- 172.21.15.3:0/133712449 >> 172.21.15.3:0/133712449 conn(0x556c98d76120 msgr2=0x556c98d80d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 -- 172.21.15.3:0/133712449 shutdown_connections 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 -- 172.21.15.3:0/133712449 wait complete. 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 -- start start 2024-08-01T04:18:05.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.914+0000 7fab54ffd740 1 Processor -- start 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d8d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d88880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d84d70 0x556c98d88e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1adfb0 con 0x556c98d79b90 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1af420 con 0x556c98d84d70 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ae680 con 0x556c98d7a4b0 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab54ffd740 10 client.0 _ll_register_callbacks cb 0x7fab340d2d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d8d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab4b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d8d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55328/0 (socket says 172.21.15.3:55328) 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab4b400640 1 -- 172.21.15.3:0/2808230124 learned_addr learned my addr 172.21.15.3:0/2808230124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.915+0000 7fab4aa00640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d88880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d84d70 0x556c98d88e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 -- 172.21.15.3:0/2808230124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 msgr2=0x556c98d88880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d88880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 -- 172.21.15.3:0/2808230124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 msgr2=0x556c98d8d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d8d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 -- 172.21.15.3:0/2808230124 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c98d799c0 con 0x556c98d84d70 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4aa00640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d88880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab4be00640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d84d70 0x556c98d88e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab3c02a920 tx=0x7fab3c032fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4a07c9bf62b3346 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.916+0000 7fab39600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:05.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.917+0000 7fab39600640 1 -- 172.21.15.3:0/2808230124 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035040 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.917+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4402ab60 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.917+0000 7fab39600640 1 -- 172.21.15.3:0/2808230124 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fab3c0040c0 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.917+0000 7fab39600640 1 -- 172.21.15.3:0/2808230124 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab3c067050 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.917+0000 7fab54ffd740 10 client.25625 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.917+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c98e07a00 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab39600640 1 -- 172.21.15.3:0/2808230124 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4402ab60 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab39600640 1 -- 172.21.15.3:0/2808230124 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fab3c004750 con 0x556c98d84d70 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab54ffd740 10 client.25625 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab54ffd740 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab54ffd740 20 client.25625 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab54ffd740 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98e08110 0x556c98e0a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:05.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab54ffd740 4 client.25625 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:18:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c98ce7670 con 0x556c98e08110 2024-08-01T04:18:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.918+0000 7fab4b400640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98e08110 0x556c98e0a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.919+0000 7fab4b400640 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98e08110 0x556c98e0a500 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7bd7c33d357f0d1 server_cookie=35069482b1e6b08f in_seq=0 out_seq=0 2024-08-01T04:18:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.919+0000 7fab39600640 10 client.25625 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:05.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.919+0000 7fab39600640 1 -- 172.21.15.3:0/2808230124 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c98ce7670 con 0x556c98e08110 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.919+0000 7fab39600640 10 client.25625 handle_command_reply: tid=0 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 client.25625 shutdown 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d84d70 msgr2=0x556c98d88e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d84d70 0x556c98d88e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fab3c02a920 tx=0x7fab3c032fc0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 shutdown_connections 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556c98e08110 0x556c98e0a500 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556c98d84d70 0x556c98d88e70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556c98d7a4b0 0x556c98d88880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 --2- 172.21.15.3:0/2808230124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556c98d79b90 0x556c98d8d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 >> 172.21.15.3:0/2808230124 conn(0x556c98d76120 msgr2=0x556c98d834e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab39600640 0 client.25625 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:05.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 shutdown_connections 2024-08-01T04:18:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 1 -- 172.21.15.3:0/2808230124 wait complete. 2024-08-01T04:18:05.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.920+0000 7fab54ffd740 20 client.25625 trim_cache size 0 max 16384 2024-08-01T04:18:05.925 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:18:05.925 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:18:05.925 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:18:05.925 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.921+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab300a2d70 msgr2=0x7fab300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.921+0000 7fab52800640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab300a2d70 0x7fab300a5230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x556c98d76f80 tx=0x7fab44009800 comp rx=0 tx=0).stop 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.921+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 msgr2=0x7fab4c19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.921+0000 7fab52800640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c19a8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab3400e9f0 tx=0x7fab3400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 shutdown_connections 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab4c1078c0 0x7fab4c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab300a2d70 0x7fab300a5230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab4c102260 0x7fab4c19a8e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 --2- 172.21.15.3:0/2323536317 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab4c1018c0 0x7fab4c19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:05.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 >> 172.21.15.3:0/2323536317 conn(0x7fab4c0fd620 msgr2=0x7fab4c10d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 shutdown_connections 2024-08-01T04:18:05.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:05.922+0000 7fab52800640 1 -- 172.21.15.3:0/2323536317 wait complete. 2024-08-01T04:18:06.922 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:18:06.923 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:18:06.999 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:07.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:07 smithi003 ceph-mon[32262]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:07.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:07 smithi163 ceph-mon[37383]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:07.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:07 smithi160 ceph-mon[37224]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 -- 172.21.15.3:0/2668901251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0103900 msgr2=0x7f36a0103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 --2- 172.21.15.3:0/2668901251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0103900 0x7f36a0103d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ca0 tx=0x7f368802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 -- 172.21.15.3:0/2668901251 shutdown_connections 2024-08-01T04:18:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 --2- 172.21.15.3:0/2668901251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a01042a0 0x7f36a01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 --2- 172.21.15.3:0/2668901251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0103900 0x7f36a0103d60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 --2- 172.21.15.3:0/2668901251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a0109900 0x7f36a0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.653+0000 7f36a5600640 1 -- 172.21.15.3:0/2668901251 >> 172.21.15.3:0/2668901251 conn(0x7f36a00ff640 msgr2=0x7f36a0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.654+0000 7f36a5600640 1 -- 172.21.15.3:0/2668901251 shutdown_connections 2024-08-01T04:18:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.654+0000 7f36a5600640 1 -- 172.21.15.3:0/2668901251 wait complete. 2024-08-01T04:18:07.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.654+0000 7f36a5600640 1 Processor -- start 2024-08-01T04:18:07.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 -- start start 2024-08-01T04:18:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 0x7f36a019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a01042a0 0x7f36a019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:07.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0109900 0x7f36a019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0113d00 con 0x7f36a0103900 2024-08-01T04:18:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0113b80 con 0x7f36a0109900 2024-08-01T04:18:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f36a5600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0113e80 con 0x7f36a01042a0 2024-08-01T04:18:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f369fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 0x7f36a019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:07.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.655+0000 7f369fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 0x7f36a019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55336/0 (socket says 172.21.15.3:55336) 2024-08-01T04:18:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369fe00640 1 -- 172.21.15.3:0/2865822115 learned_addr learned my addr 172.21.15.3:0/2865822115 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369f400640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a01042a0 0x7f36a019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f36a4c00640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0109900 0x7f36a019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:07.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369fe00640 1 -- 172.21.15.3:0/2865822115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a01042a0 msgr2=0x7f36a019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369fe00640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a01042a0 0x7f36a019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369fe00640 1 -- 172.21.15.3:0/2865822115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0109900 msgr2=0x7f36a019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369fe00640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0109900 0x7f36a019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.656+0000 7f369fe00640 1 -- 172.21.15.3:0/2865822115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a019f1b0 con 0x7f36a0103900 2024-08-01T04:18:07.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.657+0000 7f369fe00640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 0x7f36a019a200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f369000e9f0 tx=0x7f369000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.657+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000cd80 con 0x7f36a0103900 2024-08-01T04:18:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.657+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690005500 con 0x7f36a0103900 2024-08-01T04:18:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.657+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690010690 con 0x7f36a0103900 2024-08-01T04:18:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.657+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a019f4a0 con 0x7f36a0103900 2024-08-01T04:18:07.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.657+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01a6d80 con 0x7f36a0103900 2024-08-01T04:18:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.658+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a0105040 con 0x7f36a0103900 2024-08-01T04:18:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.663+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f36900046c0 con 0x7f36a0103900 2024-08-01T04:18:07.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.665+0000 7f369cc00640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f36800a2d70 0x7f36800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.665+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f36900cf840 con 0x7f36a0103900 2024-08-01T04:18:07.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.665+0000 7f369f400640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f36800a2d70 0x7f36800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:07.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.666+0000 7f369f400640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f36800a2d70 0x7f36800a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36a0101ca0 tx=0x7f3688031000 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:07.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.670+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f369009c3c0 con 0x7f36a0103900 2024-08-01T04:18:07.712 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:07.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.898+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36a0103d60 con 0x7f36a0103900 2024-08-01T04:18:07.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.900+0000 7f369cc00640 1 -- 172.21.15.3:0/2865822115 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7f369009bcc0 con 0x7f36a0103900 2024-08-01T04:18:07.901 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:18:07.901 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:18:07.902 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:18:07.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.905+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f36800a2d70 msgr2=0x7f36800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:07.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.905+0000 7f36a5600640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f36800a2d70 0x7f36800a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36a0101ca0 tx=0x7f3688031000 comp rx=0 tx=0).stop 2024-08-01T04:18:07.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.905+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 msgr2=0x7f36a019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:07.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.905+0000 7f36a5600640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 0x7f36a019a200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f369000e9f0 tx=0x7f369000edf0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 shutdown_connections 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36a0109900 0x7f36a019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f36800a2d70 0x7f36800a5230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a01042a0 0x7f36a019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 --2- 172.21.15.3:0/2865822115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36a0103900 0x7f36a019a200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 >> 172.21.15.3:0/2865822115 conn(0x7f36a00ff640 msgr2=0x7f36a00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 shutdown_connections 2024-08-01T04:18:07.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:07.906+0000 7f36a5600640 1 -- 172.21.15.3:0/2865822115 wait complete. 2024-08-01T04:18:08.464 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:08 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2865822115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 -- 172.21.15.3:0/3418263659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 msgr2=0x7fa1ac103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 --2- 172.21.15.3:0/3418263659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac103d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa198009860 tx=0x7fa19802a340 comp rx=0 tx=0).stop 2024-08-01T04:18:08.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 -- 172.21.15.3:0/3418263659 shutdown_connections 2024-08-01T04:18:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 --2- 172.21.15.3:0/3418263659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac104290 0x7fa1ac1107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 --2- 172.21.15.3:0/3418263659 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac103d50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 --2- 172.21.15.3:0/3418263659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac1098f0 0x7fa1ac109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 -- 172.21.15.3:0/3418263659 >> 172.21.15.3:0/3418263659 conn(0x7fa1ac0ff5d0 msgr2=0x7fa1ac1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 -- 172.21.15.3:0/3418263659 shutdown_connections 2024-08-01T04:18:08.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.489+0000 7fa1abe00640 1 -- 172.21.15.3:0/3418263659 wait complete. 2024-08-01T04:18:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.490+0000 7fa1abe00640 1 Processor -- start 2024-08-01T04:18:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.490+0000 7fa1abe00640 1 -- start start 2024-08-01T04:18:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:08.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 0x7fa1ac19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac1098f0 0x7fa1ac19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1abe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac113cb0 con 0x7fa1ac104290 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1abe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac113b30 con 0x7fa1ac1038f0 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1abe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac113e30 con 0x7fa1ac1098f0 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 0x7fa1ac19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 0x7fa1ac19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55358/0 (socket says 172.21.15.3:55358) 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1aa000640 1 -- 172.21.15.3:0/2971788598 learned_addr learned my addr 172.21.15.3:0/2971788598 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1aaa00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:08.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.491+0000 7fa1ab400640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac1098f0 0x7fa1ac19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aa000640 1 -- 172.21.15.3:0/2971788598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac1098f0 msgr2=0x7fa1ac19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aa000640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac1098f0 0x7fa1ac19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aa000640 1 -- 172.21.15.3:0/2971788598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 msgr2=0x7fa1ac19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aa000640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aa000640 1 -- 172.21.15.3:0/2971788598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac19f1a0 con 0x7fa1ac104290 2024-08-01T04:18:08.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aaa00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1aa000640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 0x7fa1ac19a730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa198035040 tx=0x7fa198033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.492+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198037040 con 0x7fa1ac104290 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.493+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa198040070 con 0x7fa1ac104290 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.493+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198003950 con 0x7fa1ac104290 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.493+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac19f430 con 0x7fa1ac104290 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.493+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac19f8c0 con 0x7fa1ac104290 2024-08-01T04:18:08.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.494+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac103d50 con 0x7fa1ac104290 2024-08-01T04:18:08.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.496+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa1980040e0 con 0x7fa1ac104290 2024-08-01T04:18:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.497+0000 7fa1a3400640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa1900a2d70 0x7fa1900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.497+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa1980ef2f0 con 0x7fa1ac104290 2024-08-01T04:18:08.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.498+0000 7fa1aaa00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa1900a2d70 0x7fa1900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:08.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.498+0000 7fa1aaa00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa1900a2d70 0x7fa1900a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0096c0 tx=0x7fa19c009260 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:08.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.502+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa1980bbf80 con 0x7fa1ac104290 2024-08-01T04:18:08.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:08 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2865822115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:08.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:08 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2865822115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:08.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.740+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1ac02d5e0 con 0x7fa1ac104290 2024-08-01T04:18:08.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.743+0000 7fa1a3400640 1 -- 172.21.15.3:0/2971788598 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11908 (secure 0 0 0) 0x7fa1980bb880 con 0x7fa1ac104290 2024-08-01T04:18:08.744 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:18:08.744 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-08-01T04:16:17:843640+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:16:17.179289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:18:08.744 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-08-01T04:18:08.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa1900a2d70 msgr2=0x7fa1900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:08.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa1900a2d70 0x7fa1900a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0096c0 tx=0x7fa19c009260 comp rx=0 tx=0).stop 2024-08-01T04:18:08.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 msgr2=0x7fa1ac19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:08.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 0x7fa1ac19a730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa198035040 tx=0x7fa198033b60 comp rx=0 tx=0).stop 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 shutdown_connections 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa1900a2d70 0x7fa1900a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac1098f0 0x7fa1ac19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1ac104290 0x7fa1ac19a730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 --2- 172.21.15.3:0/2971788598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1ac1038f0 0x7fa1ac19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 >> 172.21.15.3:0/2971788598 conn(0x7fa1ac0ff5d0 msgr2=0x7fa1ac0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:08.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 shutdown_connections 2024-08-01T04:18:08.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:08.746+0000 7fa1abe00640 1 -- 172.21.15.3:0/2971788598 wait complete. 2024-08-01T04:18:09.085 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db --set-id 2dc67b78 --release --await' 2024-08-01T04:18:09.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db --set-id 2dc67b78 --release --await 2024-08-01T04:18:09.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:09 smithi003 ceph-mon[32262]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-01T04:18:09.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:09 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2971788598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:09.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:09 smithi163 ceph-mon[37383]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-01T04:18:09.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:09 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2971788598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:09 smithi160 ceph-mon[37224]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-01T04:18:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:09 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2971788598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:09.878 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:18:10.043 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 -- 172.21.15.3:0/867437014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e281018b0 msgr2=0x7f3e28101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/867437014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e281018b0 0x7f3e28101d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009930 tx=0x7f3e1002a420 comp rx=0 tx=0).stop 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 -- 172.21.15.3:0/867437014 shutdown_connections 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/867437014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28102250 0x7f3e2810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/867437014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e281018b0 0x7f3e28101d10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/867437014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281078b0 0x7f3e28107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.681+0000 7f3e2fc00640 1 -- 172.21.15.3:0/867437014 >> 172.21.15.3:0/867437014 conn(0x7f3e280fd5f0 msgr2=0x7f3e280ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.682+0000 7f3e2fc00640 1 -- 172.21.15.3:0/867437014 shutdown_connections 2024-08-01T04:18:10.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.682+0000 7f3e2fc00640 1 -- 172.21.15.3:0/867437014 wait complete. 2024-08-01T04:18:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.683+0000 7f3e2fc00640 1 Processor -- start 2024-08-01T04:18:10.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.683+0000 7f3e2fc00640 1 -- start start 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2fc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281018b0 0x7f3e2819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2fc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 0x7f3e2819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2fc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e281078b0 0x7f3e2819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2fc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28111cb0 con 0x7f3e28102250 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2fc00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28111b30 con 0x7f3e281078b0 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2fc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28111e30 con 0x7f3e281018b0 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 0x7f3e2819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 0x7f3e2819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55384/0 (socket says 172.21.15.3:55384) 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e1fe00640 1 -- 172.21.15.3:0/291432873 learned_addr learned my addr 172.21.15.3:0/291432873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:10.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2d200640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281018b0 0x7f3e2819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.684+0000 7f3e2dc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e281078b0 0x7f3e2819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2d200640 1 -- 172.21.15.3:0/291432873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e281078b0 msgr2=0x7f3e2819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2d200640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e281078b0 0x7f3e2819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2d200640 1 -- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 msgr2=0x7f3e2819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2d200640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 0x7f3e2819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2d200640 1 -- 172.21.15.3:0/291432873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2819f3a0 con 0x7f3e281018b0 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 0x7f3e2819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:10.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2d200640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281018b0 0x7f3e2819a3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800e9f0 tx=0x7f3e1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e26000640 1 -- 172.21.15.3:0/291432873 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1800ccc0 con 0x7f3e281018b0 2024-08-01T04:18:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e26000640 1 -- 172.21.15.3:0/291432873 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e18010910 con 0x7f3e281018b0 2024-08-01T04:18:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.685+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2819f690 con 0x7f3e281018b0 2024-08-01T04:18:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.686+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e281a6f70 con 0x7f3e281018b0 2024-08-01T04:18:10.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.686+0000 7f3e26000640 1 -- 172.21.15.3:0/291432873 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18018960 con 0x7f3e281018b0 2024-08-01T04:18:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.689+0000 7f3e3281e740 1 Processor -- start 2024-08-01T04:18:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.689+0000 7f3e26000640 1 -- 172.21.15.3:0/291432873 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f3e18010ac0 con 0x7f3e281018b0 2024-08-01T04:18:10.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 -- start start 2024-08-01T04:18:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5eb230 0x56345e5ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56345e545e30 con 0x56345e5e0970 2024-08-01T04:18:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56345e4ddd40 con 0x56345e5eb230 2024-08-01T04:18:10.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e3281e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56345e541df0 con 0x56345e5e0050 2024-08-01T04:18:10.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5eb230 0x56345e5ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5eacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55396/0 (socket says 172.21.15.3:55396) 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.690+0000 7f3e2d200640 1 -- 172.21.15.3:0/3963854291 learned_addr learned my addr 172.21.15.3:0/3963854291 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.691+0000 7f3e2dc00640 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.691+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3963854291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 msgr2=0x56345e5e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.691+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.691+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3963854291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 msgr2=0x56345e5eacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.691+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5eacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.691+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3963854291 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56345e255580 con 0x56345e5eb230 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e26000640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e26000640 1 -- 172.21.15.3:0/291432873 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3e180cf610 con 0x7f3e281018b0 2024-08-01T04:18:10.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5eb230 0x56345e5ed620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009900 tx=0x7f3e10005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc73f10c6e160de server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:10.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e2dc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e1e000640 1 -- 172.21.15.3:0/3963854291 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10035040 con 0x56345e5eb230 2024-08-01T04:18:10.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e1e000640 1 -- 172.21.15.3:0/3963854291 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10031830 con 0x56345e5eb230 2024-08-01T04:18:10.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.692+0000 7f3e1e000640 1 -- 172.21.15.3:0/3963854291 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100047a0 con 0x56345e5eb230 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.693+0000 7f3e3281e740 1 -- 172.21.15.3:0/3963854291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5eb230 msgr2=0x56345e5ed620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.693+0000 7f3e3281e740 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5eb230 0x56345e5ed620 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009900 tx=0x7f3e10005dc0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.693+0000 7f3e2dc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005fd0 tx=0x7f3e20009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 -- 172.21.15.3:0/3963854291 shutdown_connections 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5eb230 0x56345e5ed620 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5eacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 --2- 172.21.15.3:0/3963854291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 -- 172.21.15.3:0/3963854291 >> 172.21.15.3:0/3963854291 conn(0x56345e5dc5e0 msgr2=0x56345e5dc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 -- 172.21.15.3:0/3963854291 shutdown_connections 2024-08-01T04:18:10.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 -- 172.21.15.3:0/3963854291 wait complete. 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.694+0000 7f3e3281e740 1 -- start start 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 Processor -- start 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 0x56345e5e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5eb230 0x56345e5e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56345e5ede00 con 0x56345e5eb230 2024-08-01T04:18:10.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56345e500e70 con 0x56345e5e0970 2024-08-01T04:18:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56345e5edf80 con 0x56345e5e0050 2024-08-01T04:18:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.695+0000 7f3e3281e740 10 client.0 _ll_register_callbacks cb 0x56345e4eff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 0x56345e5e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e2dc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5eb230 0x56345e5e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 0x56345e5e8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:53234/0 (socket says 172.21.15.3:53234) 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 -- 172.21.15.3:0/1278901344 learned_addr learned my addr 172.21.15.3:0/1278901344 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e2d200640 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 -- 172.21.15.3:0/1278901344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 msgr2=0x56345e5e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 -- 172.21.15.3:0/1278901344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5eb230 msgr2=0x56345e5e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5eb230 0x56345e5e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.696+0000 7f3e1fe00640 1 -- 172.21.15.3:0/1278901344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56345e31f960 con 0x56345e5e0970 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 0x56345e5e8570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e1002abc0 tx=0x7f3e10031b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777749e7934e9634 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e16a00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e16a00640 1 -- 172.21.15.3:0/1278901344 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1004e020 con 0x56345e5e0970 2024-08-01T04:18:10.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e16a00640 1 -- 172.21.15.3:0/1278901344 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3e10004250 con 0x56345e5e0970 2024-08-01T04:18:10.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e10031830 con 0x56345e5e0970 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e16a00640 1 -- 172.21.15.3:0/1278901344 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e10059050 con 0x56345e5e0970 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e3281e740 10 client.25649 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.697+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56345e5f9dc0 con 0x56345e5e0970 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e16a00640 1 -- 172.21.15.3:0/1278901344 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10031830 con 0x56345e5e0970 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e16a00640 1 -- 172.21.15.3:0/1278901344 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3e100466f0 con 0x56345e5e0970 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e3281e740 10 client.25649 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e3281e740 10 client.25649 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e3281e740 20 client.25649 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e5fb110 0x56345e5fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e3281e740 4 client.25649 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:10.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56345e559850 con 0x56345e5fb110 2024-08-01T04:18:10.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.698+0000 7f3e2d200640 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e5fb110 0x56345e5fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.699+0000 7f3e2d200640 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e5fb110 0x56345e5fd500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db40db1410de012b server_cookie=70c99a46c43aa976 in_seq=0 out_seq=0 2024-08-01T04:18:10.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.699+0000 7f3e16a00640 10 client.25649 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.699+0000 7f3e16a00640 1 -- 172.21.15.3:0/1278901344 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56345e559850 con 0x56345e5fb110 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e16a00640 10 client.25649 handle_command_reply: tid=0 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 client.25649 shutdown 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 msgr2=0x56345e5e8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 0x56345e5e8570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e1002abc0 tx=0x7f3e10031b00 comp rx=0 tx=0).stop 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 shutdown_connections 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e5fb110 0x56345e5fd500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5eb230 0x56345e5e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0970 0x56345e5e8570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1278901344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5e0050 0x56345e5e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e16a00640 0 client.25649 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 >> 172.21.15.3:0/1278901344 conn(0x56345e5dc5e0 msgr2=0x56345e5e2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 shutdown_connections 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 1 -- 172.21.15.3:0/1278901344 wait complete. 2024-08-01T04:18:10.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.700+0000 7f3e3281e740 20 client.25649 trim_cache size 0 max 16384 2024-08-01T04:18:10.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 Processor -- start 2024-08-01T04:18:10.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 -- start start 2024-08-01T04:18:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5e9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e602bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e603100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04011140 con 0x56345e5e0970 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0400ea10 con 0x56345e5e0050 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e3281e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0400c860 con 0x56345e5eb230 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.720+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e602bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e603100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5e9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2dc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e603100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53018/0 (socket says 172.21.15.3:53018) 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2dc00640 1 -- 172.21.15.3:0/1668857831 learned_addr learned my addr 172.21.15.3:0/1668857831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 -- 172.21.15.3:0/1668857831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 msgr2=0x56345e603100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e603100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 -- 172.21.15.3:0/1668857831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 msgr2=0x56345e602bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e602bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 -- 172.21.15.3:0/1668857831 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56345e5dd270 con 0x56345e5e0050 2024-08-01T04:18:10.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.721+0000 7f3e2d200640 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5e9fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c6a0 tx=0x7f3e180d4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a422a2552b3770e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:10.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.722+0000 7f3e1d600640 1 -- 172.21.15.3:0/1668857831 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1800ce60 con 0x56345e5e0050 2024-08-01T04:18:10.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.722+0000 7f3e1d600640 1 -- 172.21.15.3:0/1668857831 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e18010db0 con 0x56345e5e0050 2024-08-01T04:18:10.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.722+0000 7f3e1d600640 1 -- 172.21.15.3:0/1668857831 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e180ccf40 con 0x56345e5e0050 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.722+0000 7f3e3281e740 1 -- 172.21.15.3:0/1668857831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 msgr2=0x56345e5e9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.722+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5e9fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c6a0 tx=0x7f3e180d4040 comp rx=0 tx=0).stop 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 -- 172.21.15.3:0/1668857831 shutdown_connections 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e603100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e602bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 --2- 172.21.15.3:0/1668857831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5e9fa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 -- 172.21.15.3:0/1668857831 >> 172.21.15.3:0/1668857831 conn(0x56345e5dc5e0 msgr2=0x56345e5e9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 -- 172.21.15.3:0/1668857831 shutdown_connections 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 -- 172.21.15.3:0/1668857831 wait complete. 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.723+0000 7f3e3281e740 1 -- start start 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 Processor -- start 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e5e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0400ced0 con 0x56345e5e0970 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0400e340 con 0x56345e5e0050 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0400d5a0 con 0x56345e5eb230 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e3281e740 10 client.0 _ll_register_callbacks cb 0x7f3e10040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e1fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:53258/0 (socket says 172.21.15.3:53258) 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.724+0000 7f3e2d200640 1 -- 172.21.15.3:0/2722129073 learned_addr learned my addr 172.21.15.3:0/2722129073 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2dc00640 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e5e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2d200640 1 -- 172.21.15.3:0/2722129073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 msgr2=0x56345e5e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2d200640 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e5e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2d200640 1 -- 172.21.15.3:0/2722129073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 msgr2=0x56345e5e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2d200640 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2d200640 1 -- 172.21.15.3:0/2722129073 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56345e5e5510 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e2d200640 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5f36b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e180d4010 tx=0x7f3e1800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0379a25e3f7c986 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.725+0000 7f3e16a00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e16a00640 1 -- 172.21.15.3:0/2722129073 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e180b3020 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e16a00640 1 -- 172.21.15.3:0/2722129073 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3e180ccf40 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e18010db0 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e16a00640 1 -- 172.21.15.3:0/2722129073 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e180cd140 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 10 client.25655 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56345e6658e0 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e16a00640 1 -- 172.21.15.3:0/2722129073 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e18010db0 con 0x56345e5e0050 2024-08-01T04:18:10.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e16a00640 1 -- 172.21.15.3:0/2722129073 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3e180cf170 con 0x56345e5e0050 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 10 client.25655 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 10 client.25655 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 20 client.25655 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.726+0000 7f3e3281e740 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e666000 0x56345e668420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.727+0000 7f3e3281e740 4 client.25655 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2dc67b78", "release": true, "await": true}] 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.727+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2dc67b78", "release": true, "await": true}) -- 0x56345e5533a0 con 0x56345e666000 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.727+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e666000 0x56345e668420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.727+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e666000 0x56345e668420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a39423ac1960c5 server_cookie=add34ca4df4f5630 in_seq=0 out_seq=0 2024-08-01T04:18:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.727+0000 7f3e16a00640 10 client.25655 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.730+0000 7f3e16a00640 1 -- 172.21.15.3:0/2722129073 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56345e5533a0 con 0x56345e666000 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.730+0000 7f3e16a00640 10 client.25655 handle_command_reply: tid=0 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.730+0000 7f3e3281e740 1 client.25655 shutdown 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 msgr2=0x56345e5f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5f36b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e180d4010 tx=0x7f3e1800ce60 comp rx=0 tx=0).stop 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 shutdown_connections 2024-08-01T04:18:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56345e666000 0x56345e668420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56345e5eb230 0x56345e5e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56345e5e0970 0x56345e5e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 --2- 172.21.15.3:0/2722129073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56345e5e0050 0x56345e5f36b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 >> 172.21.15.3:0/2722129073 conn(0x56345e5dc5e0 msgr2=0x56345e5db780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e16a00640 0 client.25655 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 shutdown_connections 2024-08-01T04:18:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 1 -- 172.21.15.3:0/2722129073 wait complete. 2024-08-01T04:18:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.731+0000 7f3e3281e740 20 client.25655 trim_cache size 0 max 16384 2024-08-01T04:18:10.735 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:18:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-08-01T04:18:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:18:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 8, 2024-08-01T04:18:10.735 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:18:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: "2dc67b78": { 2024-08-01T04:18:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 8, 2024-08-01T04:18:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:18:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:18:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:18:10.736 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:18:10.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:18:10.737 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:18:10.737 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.4, 2024-08-01T04:18:10.737 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:18:10.737 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:18:10.737 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:18:10.738 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:18:10.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:18:10.738 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:18:10.738 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:10.738 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:10.738 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:10.739 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:10.739 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:10.739 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:18:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.732+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2d70 msgr2=0x7f3e0c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.732+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005fd0 tx=0x7f3e20009500 comp rx=0 tx=0).stop 2024-08-01T04:18:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.732+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281018b0 msgr2=0x7f3e2819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.732+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281018b0 0x7f3e2819a3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800e9f0 tx=0x7f3e1800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 shutdown_connections 2024-08-01T04:18:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e281078b0 0x7f3e2819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28102250 0x7f3e2819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 --2- 172.21.15.3:0/291432873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e281018b0 0x7f3e2819a3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 >> 172.21.15.3:0/291432873 conn(0x7f3e280fd5f0 msgr2=0x7f3e2810cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 shutdown_connections 2024-08-01T04:18:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:10.733+0000 7f3e2fc00640 1 -- 172.21.15.3:0/291432873 wait complete. 2024-08-01T04:18:10.747 INFO:tasks.workunit.client.0.smithi003.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-08-01T04:18:10.749 INFO:tasks.workunit.client.0.smithi003.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-08-01T04:18:10.750 INFO:tasks.workunit.client.0.smithi003.stderr:+ trap -- retry EXIT 2024-08-01T04:18:10.750 INFO:tasks.workunit.client.0.smithi003.stderr:+ rm -rf ffsb 2024-08-01T04:18:10.752 INFO:tasks.workunit.client.0.smithi003.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-08-01T04:18:10.760 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into 'ffsb'... 2024-08-01T04:18:11.115 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:11.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:11 smithi163 ceph-mon[37383]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:11.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:11 smithi003 ceph-mon[32262]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:11.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:11 smithi160 ceph-mon[37224]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-08-01T04:18:11.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2dc67b78', total seconds elapsed: 30.3 2024-08-01T04:18:11.990 INFO:tasks.workunit.client.0.smithi003.stderr:+ trap - EXIT 2024-08-01T04:18:11.990 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd ffsb 2024-08-01T04:18:11.990 INFO:tasks.workunit.client.0.smithi003.stderr:+ ./configure 2024-08-01T04:18:12.349 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-01T04:18:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 -- 172.21.15.3:0/2754319520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 msgr2=0x7fb69c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 --2- 172.21.15.3:0/2754319520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c100a90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009900 tx=0x7fb68c02a480 comp rx=0 tx=0).stop 2024-08-01T04:18:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 -- 172.21.15.3:0/2754319520 shutdown_connections 2024-08-01T04:18:12.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 --2- 172.21.15.3:0/2754319520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb69c105ac0 0x7fb69c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 --2- 172.21.15.3:0/2754319520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb69c100fd0 0x7fb69c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 --2- 172.21.15.3:0/2754319520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c100a90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 -- 172.21.15.3:0/2754319520 >> 172.21.15.3:0/2754319520 conn(0x7fb69c0fc360 msgr2=0x7fb69c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 -- 172.21.15.3:0/2754319520 shutdown_connections 2024-08-01T04:18:12.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.404+0000 7fb69be00640 1 -- 172.21.15.3:0/2754319520 wait complete. 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.405+0000 7fb69be00640 1 Processor -- start 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.405+0000 7fb69be00640 1 -- start start 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb69c100fd0 0x7fb69c19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb69c105ac0 0x7fb69c1a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69be00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1180b0 con 0x7fb69c100fd0 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69be00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c117f30 con 0x7fb69c1006b0 2024-08-01T04:18:12.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69be00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c118230 con 0x7fb69c105ac0 2024-08-01T04:18:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb69c100fd0 0x7fb69c19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c19e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52476/0 (socket says 172.21.15.3:52476) 2024-08-01T04:18:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69aa00640 1 -- 172.21.15.3:0/500314010 learned_addr learned my addr 172.21.15.3:0/500314010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.406+0000 7fb69b400640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb69c105ac0 0x7fb69c1a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.407+0000 7fb69aa00640 1 -- 172.21.15.3:0/500314010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb69c105ac0 msgr2=0x7fb69c1a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.407+0000 7fb69aa00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb69c105ac0 0x7fb69c1a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.407+0000 7fb69aa00640 1 -- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb69c100fd0 msgr2=0x7fb69c19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.407+0000 7fb69aa00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb69c100fd0 0x7fb69c19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.407+0000 7fb69aa00640 1 -- 172.21.15.3:0/500314010 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69c1a3540 con 0x7fb69c1006b0 2024-08-01T04:18:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.407+0000 7fb69aa00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c19e590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009230 tx=0x7fb68c032d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:12.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.408+0000 7fb6a0a00640 1 -- 172.21.15.3:0/500314010 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c03f030 con 0x7fb69c1006b0 2024-08-01T04:18:12.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.408+0000 7fb6a0a00640 1 -- 172.21.15.3:0/500314010 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68c003ce0 con 0x7fb69c1006b0 2024-08-01T04:18:12.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.408+0000 7fb6a0a00640 1 -- 172.21.15.3:0/500314010 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c035070 con 0x7fb69c1006b0 2024-08-01T04:18:12.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.408+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69c1a37d0 con 0x7fb69c1006b0 2024-08-01T04:18:12.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.408+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69c1a3c60 con 0x7fb69c1006b0 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.411+0000 7fb6a4a32740 1 Processor -- start 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.411+0000 7fb6a0a00640 1 -- 172.21.15.3:0/500314010 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb68c042030 con 0x7fb69c1006b0 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.411+0000 7fb6a4a32740 1 -- start start 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b176e0 0x55fc58b17ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b22380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b228c0 0x55fc58b24cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fc58a7cbf0 con 0x55fc58b176e0 2024-08-01T04:18:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fc58a14b20 con 0x55fc58b18000 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fc58a78bb0 con 0x55fc58b228c0 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b22380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb69b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b176e0 0x55fc58b17ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b22380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52488/0 (socket says 172.21.15.3:52488) 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb69aa00640 1 -- 172.21.15.3:0/1280660447 learned_addr learned my addr 172.21.15.3:0/1280660447 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.412+0000 7fb69a000640 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b228c0 0x55fc58b24cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb69aa00640 1 -- 172.21.15.3:0/1280660447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b228c0 msgr2=0x55fc58b24cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb69aa00640 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b228c0 0x55fc58b24cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb69aa00640 1 -- 172.21.15.3:0/1280660447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b176e0 msgr2=0x55fc58b17ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb69aa00640 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b176e0 0x55fc58b17ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb69aa00640 1 -- 172.21.15.3:0/1280660447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc58a1b7f0 con 0x55fc58b18000 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb69aa00640 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b22380 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0ecf70 tx=0x7fb68c0f5cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e010259707f3d45 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb687e00640 1 -- 172.21.15.3:0/1280660447 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0fe070 con 0x55fc58b18000 2024-08-01T04:18:12.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb687e00640 1 -- 172.21.15.3:0/1280660447 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68c0f7c90 con 0x55fc58b18000 2024-08-01T04:18:12.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.413+0000 7fb687e00640 1 -- 172.21.15.3:0/1280660447 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0f9660 con 0x55fc58b18000 2024-08-01T04:18:12.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a0a00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb6800a2d70 0x7fb6800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 -- 172.21.15.3:0/1280660447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 msgr2=0x55fc58b22380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b22380 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0ecf70 tx=0x7fb68c0f5cb0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a0a00640 1 -- 172.21.15.3:0/500314010 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb68c0ef980 con 0x7fb69c1006b0 2024-08-01T04:18:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb69b400640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb6800a2d70 0x7fb6800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 -- 172.21.15.3:0/1280660447 shutdown_connections 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b228c0 0x55fc58b24cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b22380 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/1280660447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b176e0 0x55fc58b17ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.414+0000 7fb6a4a32740 1 -- 172.21.15.3:0/1280660447 >> 172.21.15.3:0/1280660447 conn(0x55fc58b13c70 msgr2=0x55fc58b14050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.415+0000 7fb6a4a32740 1 -- 172.21.15.3:0/1280660447 shutdown_connections 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.415+0000 7fb6a4a32740 1 -- 172.21.15.3:0/1280660447 wait complete. 2024-08-01T04:18:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.415+0000 7fb69b400640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb6800a2d70 0x7fb6800a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb694005fd0 tx=0x7fb694009600 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.415+0000 7fb6a4a32740 1 -- start start 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 Processor -- start 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b18000 0x55fc58b1fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 0x55fc58b20150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fc58b25250 con 0x55fc58b18000 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fc58a37cf0 con 0x55fc58b228c0 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fc58b253d0 con 0x55fc58b176e0 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb6a4a32740 10 client.0 _ll_register_callbacks cb 0x55fc58a23e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.416+0000 7fb69b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 0x55fc58b20150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b18000 0x55fc58b1fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 0x55fc58b20150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52496/0 (socket says 172.21.15.3:52496) 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 -- 172.21.15.3:0/2505382486 learned_addr learned my addr 172.21.15.3:0/2505382486 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 -- 172.21.15.3:0/2505382486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 msgr2=0x55fc58b1f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b1f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 -- 172.21.15.3:0/2505382486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b18000 msgr2=0x55fc58b1fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b18000 0x55fc58b1fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 -- 172.21.15.3:0/2505382486 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc587ee3c0 con 0x55fc58b228c0 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb69b400640 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 0x55fc58b20150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb694013fd0 tx=0x7fb694004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b563909961c9201f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.417+0000 7fb684c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb684c00640 1 -- 172.21.15.3:0/2505382486 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69401f8f0 con 0x55fc58b228c0 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb68c0f7c90 con 0x55fc58b228c0 2024-08-01T04:18:12.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb684c00640 1 -- 172.21.15.3:0/2505382486 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb69401fa90 con 0x55fc58b228c0 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb684c00640 1 -- 172.21.15.3:0/2505382486 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb69401fc90 con 0x55fc58b228c0 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb6a4a32740 10 client.25667 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc58b30e90 con 0x55fc58b228c0 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb684c00640 1 -- 172.21.15.3:0/2505382486 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68c0f7c90 con 0x55fc58b228c0 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.418+0000 7fb684c00640 1 -- 172.21.15.3:0/2505382486 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb69400f040 con 0x55fc58b228c0 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb6a4a32740 10 client.25667 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb6a4a32740 10 client.25667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb6a4a32740 20 client.25667 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b322f0 0x55fc58b34730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb6a4a32740 4 client.25667 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:12.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc58a8a160 con 0x55fc58b322f0 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.419+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b322f0 0x55fc58b34730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.420+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b322f0 0x55fc58b34730 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e7d723be1b91074 server_cookie=6a6eeee48851b0f8 in_seq=0 out_seq=0 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.420+0000 7fb684c00640 10 client.25667 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.420+0000 7fb684c00640 1 -- 172.21.15.3:0/2505382486 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc58a8a160 con 0x55fc58b322f0 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.420+0000 7fb684c00640 10 client.25667 handle_command_reply: tid=0 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.420+0000 7fb6a4a32740 1 client.25667 shutdown 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 msgr2=0x55fc58b20150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 0x55fc58b20150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb694013fd0 tx=0x7fb694004a40 comp rx=0 tx=0).stop 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 shutdown_connections 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b228c0 0x55fc58b20150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b18000 0x55fc58b1fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b322f0 0x55fc58b34730 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2505382486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 >> 172.21.15.3:0/2505382486 conn(0x55fc58b13c70 msgr2=0x55fc58b199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb684c00640 0 client.25667 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:18:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 shutdown_connections 2024-08-01T04:18:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2505382486 wait complete. 2024-08-01T04:18:12.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.421+0000 7fb6a4a32740 20 client.25667 trim_cache size 0 max 16384 2024-08-01T04:18:12.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 Processor -- start 2024-08-01T04:18:12.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 -- start start 2024-08-01T04:18:12.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 0x55fc58b19890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b18000 0x55fc58b3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b3a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1b63e0 con 0x55fc58b228c0 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1b3cb0 con 0x55fc58b176e0 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.436+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1b1b00 con 0x55fc58b18000 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b3a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 0x55fc58b19890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 0x55fc58b19890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52506/0 (socket says 172.21.15.3:52506) 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b3a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60332/0 (socket says 172.21.15.3:60332) 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 -- 172.21.15.3:0/2146989954 learned_addr learned my addr 172.21.15.3:0/2146989954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:12.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69a000640 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b18000 0x55fc58b3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 -- 172.21.15.3:0/2146989954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b18000 msgr2=0x55fc58b3a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b18000 0x55fc58b3a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 -- 172.21.15.3:0/2146989954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 msgr2=0x55fc58b3a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b3a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 -- 172.21.15.3:0/2146989954 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc58b14900 con 0x55fc58b176e0 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.437+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 0x55fc58b19890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb68c005ab0 tx=0x7fb68c03b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c998836d516ea7b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb687400640 1 -- 172.21.15.3:0/2146989954 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c049070 con 0x55fc58b176e0 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb687400640 1 -- 172.21.15.3:0/2146989954 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68c0bbeb0 con 0x55fc58b176e0 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb687400640 1 -- 172.21.15.3:0/2146989954 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0fe070 con 0x55fc58b176e0 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2146989954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 msgr2=0x55fc58b19890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 0x55fc58b19890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb68c005ab0 tx=0x7fb68c03b6b0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2146989954 shutdown_connections 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b3a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b18000 0x55fc58b3a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2146989954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b176e0 0x55fc58b19890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2146989954 >> 172.21.15.3:0/2146989954 conn(0x55fc58b13c70 msgr2=0x55fc58b19180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.438+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2146989954 shutdown_connections 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.439+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2146989954 wait complete. 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.439+0000 7fb6a4a32740 1 -- start start 2024-08-01T04:18:12.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.439+0000 7fb6a4a32740 1 Processor -- start 2024-08-01T04:18:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b2abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b1c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1b2170 con 0x55fc58b228c0 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1b35e0 con 0x55fc58b18000 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb69c1b2840 con 0x55fc58b176e0 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb6a4a32740 10 client.0 _ll_register_callbacks cb 0x7fb694003c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb69b400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b2abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb69aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b2abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:47438/0 (socket says 172.21.15.3:47438) 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb69aa00640 1 -- 172.21.15.3:0/2975588458 learned_addr learned my addr 172.21.15.3:0/2975588458 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb69a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b1c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.440+0000 7fb69aa00640 1 -- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 msgr2=0x55fc58b1c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b1c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69aa00640 1 -- 172.21.15.3:0/2975588458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 msgr2=0x55fc58b1c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:12.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b1c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69aa00640 1 -- 172.21.15.3:0/2975588458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc58b2e4b0 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69a000640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b1c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69b400640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb69aa00640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b2abc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0b8310 tx=0x7fb68c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78f81c63ee7847b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb684c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb684c00640 1 -- 172.21.15.3:0/2975588458 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c049070 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb684c00640 1 -- 172.21.15.3:0/2975588458 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb68c03b460 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb68c0bbeb0 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb684c00640 1 -- 172.21.15.3:0/2975588458 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb68c0fe050 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb6a4a32740 10 client.25663 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.441+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc58b29360 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb684c00640 1 -- 172.21.15.3:0/2975588458 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68c0bbeb0 con 0x55fc58b176e0 2024-08-01T04:18:12.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb684c00640 1 -- 172.21.15.3:0/2975588458 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fb68c0b8520 con 0x55fc58b176e0 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb6a4a32740 10 client.25663 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb6a4a32740 10 client.25663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb6a4a32740 20 client.25663 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b9cfd0 0x55fc58b9f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb6a4a32740 4 client.25663 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc58a84670 con 0x55fc58b9cfd0 2024-08-01T04:18:12.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.442+0000 7fb69a000640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b9cfd0 0x55fc58b9f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.443+0000 7fb69a000640 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b9cfd0 0x55fc58b9f3c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51cc34b3a40070e3 server_cookie=3818a34e52b11d08 in_seq=0 out_seq=0 2024-08-01T04:18:12.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.443+0000 7fb684c00640 10 client.25663 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.443+0000 7fb684c00640 1 -- 172.21.15.3:0/2975588458 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc58a84670 con 0x55fc58b9cfd0 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.443+0000 7fb684c00640 10 client.25663 handle_command_reply: tid=0 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.443+0000 7fb6a4a32740 1 client.25663 shutdown 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 msgr2=0x55fc58b2abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b2abc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0b8310 tx=0x7fb68c02a480 comp rx=0 tx=0).stop 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 shutdown_connections 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc58b228c0 0x55fc58b1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fc58b18000 0x55fc58b1c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fc58b9cfd0 0x55fc58b9f3c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 --2- 172.21.15.3:0/2975588458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fc58b176e0 0x55fc58b2abc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb684c00640 0 client.25663 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 >> 172.21.15.3:0/2975588458 conn(0x55fc58b13c70 msgr2=0x55fc58b199a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:12.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 shutdown_connections 2024-08-01T04:18:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 1 -- 172.21.15.3:0/2975588458 wait complete. 2024-08-01T04:18:12.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.444+0000 7fb6a4a32740 20 client.25663 trim_cache size 0 max 16384 2024-08-01T04:18:12.449 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb6800a2d70 msgr2=0x7fb6800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb6800a2d70 0x7fb6800a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb694005fd0 tx=0x7fb694009600 comp rx=0 tx=0).stop 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 msgr2=0x7fb69c19e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c19e590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009230 tx=0x7fb68c032d90 comp rx=0 tx=0).stop 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 shutdown_connections 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb6800a2d70 0x7fb6800a5230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb69c105ac0 0x7fb69c1a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb69c100fd0 0x7fb69c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 --2- 172.21.15.3:0/500314010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c1006b0 0x7fb69c19e590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:12.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 >> 172.21.15.3:0/500314010 conn(0x7fb69c0fc360 msgr2=0x7fb69c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 shutdown_connections 2024-08-01T04:18:12.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:12.445+0000 7fb69be00640 1 -- 172.21.15.3:0/500314010 wait complete. 2024-08-01T04:18:13.288 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:18:13.365 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether build environment is sane... yes 2024-08-01T04:18:13.388 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-08-01T04:18:13.388 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gawk... gawk 2024-08-01T04:18:13.415 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether make sets $(MAKE)... yes 2024-08-01T04:18:13.490 INFO:tasks.workunit.client.0.smithi003.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-01T04:18:13.490 INFO:tasks.workunit.client.0.smithi003.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-01T04:18:13.507 INFO:tasks.workunit.client.0.smithi003.stdout:checking for style of include used by make... GNU 2024-08-01T04:18:13.512 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... gcc 2024-08-01T04:18:13.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:13 smithi163 ceph-mon[37383]: pgmap v61: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 135 KiB/s wr, 29 op/s 2024-08-01T04:18:13.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:13 smithi003 ceph-mon[32262]: pgmap v61: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 135 KiB/s wr, 29 op/s 2024-08-01T04:18:13.630 INFO:tasks.workunit.client.0.smithi003.stdout:checking for C compiler default output file name... a.out 2024-08-01T04:18:13.637 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether the C compiler works... yes 2024-08-01T04:18:13.639 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are cross compiling... no 2024-08-01T04:18:13.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:13 smithi160 ceph-mon[37224]: pgmap v61: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 135 KiB/s wr, 29 op/s 2024-08-01T04:18:13.712 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of executables... 2024-08-01T04:18:13.769 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:13.780 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of object files... o 2024-08-01T04:18:13.845 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... yes 2024-08-01T04:18:13.920 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... yes 2024-08-01T04:18:14.181 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... none needed 2024-08-01T04:18:14.364 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... gcc3 2024-08-01T04:18:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 -- 172.21.15.3:0/119902047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0109900 msgr2=0x7fbfd0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 --2- 172.21.15.3:0/119902047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0109900 0x7fbfd0109ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 -- 172.21.15.3:0/119902047 shutdown_connections 2024-08-01T04:18:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 --2- 172.21.15.3:0/119902047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd01042a0 0x7fbfd01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 --2- 172.21.15.3:0/119902047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd0103900 0x7fbfd0103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 --2- 172.21.15.3:0/119902047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0109900 0x7fbfd0109ce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 -- 172.21.15.3:0/119902047 >> 172.21.15.3:0/119902047 conn(0x7fbfd00ff600 msgr2=0x7fbfd0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:14.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 -- 172.21.15.3:0/119902047 shutdown_connections 2024-08-01T04:18:14.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.463+0000 7fbfd7000640 1 -- 172.21.15.3:0/119902047 wait complete. 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.464+0000 7fbfd7000640 1 Processor -- start 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.464+0000 7fbfd7000640 1 -- start start 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd7000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 0x7fbfd019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd7000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd01042a0 0x7fbfd019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd7000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd0109900 0x7fbfd019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd7000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0113d00 con 0x7fbfd0109900 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd7000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0113b80 con 0x7fbfd0103900 2024-08-01T04:18:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd7000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0113e80 con 0x7fbfd01042a0 2024-08-01T04:18:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd5c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 0x7fbfd019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd0109900 0x7fbfd019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd5c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 0x7fbfd019a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52526/0 (socket says 172.21.15.3:52526) 2024-08-01T04:18:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd5c00640 1 -- 172.21.15.3:0/2886364855 learned_addr learned my addr 172.21.15.3:0/2886364855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.465+0000 7fbfd5200640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd01042a0 0x7fbfd019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd5c00640 1 -- 172.21.15.3:0/2886364855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd01042a0 msgr2=0x7fbfd019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd01042a0 0x7fbfd019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd5c00640 1 -- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd0109900 msgr2=0x7fbfd019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd0109900 0x7fbfd019ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd5c00640 1 -- 172.21.15.3:0/2886364855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019f160 con 0x7fbfd0103900 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 0x7fbfd019a1b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006010 tx=0x7fbfc4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfce000640 1 -- 172.21.15.3:0/2886364855 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc403f070 con 0x7fbfd0103900 2024-08-01T04:18:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfce000640 1 -- 172.21.15.3:0/2886364855 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc4031c30 con 0x7fbfd0103900 2024-08-01T04:18:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfce000640 1 -- 172.21.15.3:0/2886364855 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4035030 con 0x7fbfd0103900 2024-08-01T04:18:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.466+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019f3f0 con 0x7fbfd0103900 2024-08-01T04:18:14.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.467+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd019f880 con 0x7fbfd0103900 2024-08-01T04:18:14.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.471+0000 7fbfdb096740 1 Processor -- start 2024-08-01T04:18:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.471+0000 7fbfdb096740 1 -- start start 2024-08-01T04:18:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.471+0000 7fbfce000640 1 -- 172.21.15.3:0/2886364855 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fbfc4042070 con 0x7fbfd0103900 2024-08-01T04:18:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.471+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba4dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba5acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfdb096740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55745b9b2bf0 con 0x55745ba588c0 2024-08-01T04:18:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfdb096740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55745b94ab20 con 0x55745ba4d6e0 2024-08-01T04:18:14.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfdb096740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55745b9aebb0 con 0x55745ba4e000 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba4dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba4dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52540/0 (socket says 172.21.15.3:52540) 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 -- 172.21.15.3:0/1244116220 learned_addr learned my addr 172.21.15.3:0/1244116220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd5200640 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba5acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 -- 172.21.15.3:0/1244116220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 msgr2=0x55745ba58380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba58380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 -- 172.21.15.3:0/1244116220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 msgr2=0x55745ba5acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba5acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd6600640 1 -- 172.21.15.3:0/1244116220 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745b9517f0 con 0x55745ba4d6e0 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.472+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.473+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba4dac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009f50 tx=0x7fbfc80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0e0f84440416dd server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.473+0000 7fbfc2a00640 1 -- 172.21.15.3:0/1244116220 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8004b20 con 0x55745ba4d6e0 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.473+0000 7fbfc2a00640 1 -- 172.21.15.3:0/1244116220 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc8004cc0 con 0x55745ba4d6e0 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.473+0000 7fbfc2a00640 1 -- 172.21.15.3:0/1244116220 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8005020 con 0x55745ba4d6e0 2024-08-01T04:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.473+0000 7fbfce000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbfb80a2d70 0x7fbfb80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfce000640 1 -- 172.21.15.3:0/2886364855 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbfc40efa50 con 0x7fbfd0103900 2024-08-01T04:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbfb80a2d70 0x7fbfb80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 -- 172.21.15.3:0/1244116220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 msgr2=0x55745ba4dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba4dac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009f50 tx=0x7fbfc80118d0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 -- 172.21.15.3:0/1244116220 shutdown_connections 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba5acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1244116220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba4dac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.474+0000 7fbfdb096740 1 -- 172.21.15.3:0/1244116220 >> 172.21.15.3:0/1244116220 conn(0x55745ba49c70 msgr2=0x55745ba4a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.475+0000 7fbfdb096740 1 -- 172.21.15.3:0/1244116220 shutdown_connections 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.475+0000 7fbfdb096740 1 -- 172.21.15.3:0/1244116220 wait complete. 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.475+0000 7fbfd5c00640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbfb80a2d70 0x7fbfb80a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55745ba4aad0 tx=0x7fbfc40058b0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.475+0000 7fbfdb096740 1 -- start start 2024-08-01T04:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.475+0000 7fbfdb096740 1 Processor -- start 2024-08-01T04:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba55740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba55c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55745ba5b490 con 0x55745ba4e000 2024-08-01T04:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55745b96dcf0 con 0x55745ba588c0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55745ba5b610 con 0x55745ba4d6e0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfdb096740 10 client.0 _ll_register_callbacks cb 0x55745b959e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba55740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfd5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba55c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba55740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:47472/0 (socket says 172.21.15.3:47472) 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfd6600640 1 -- 172.21.15.3:0/2182494752 learned_addr learned my addr 172.21.15.3:0/2182494752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.476+0000 7fbfd6600640 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfd6600640 1 -- 172.21.15.3:0/2182494752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 msgr2=0x55745ba561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfd6600640 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfd6600640 1 -- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 msgr2=0x55745ba55c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfd6600640 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba55c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfd6600640 1 -- 172.21.15.3:0/2182494752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745b7243c0 con 0x55745ba4d6e0 2024-08-01T04:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfd6600640 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba55740 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8004640 tx=0x7fbfc8004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622a746999d94fef server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.477+0000 7fbfb7400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfb7400640 1 -- 172.21.15.3:0/2182494752 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc802c020 con 0x55745ba4d6e0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc8004cc0 con 0x55745ba4d6e0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfb7400640 1 -- 172.21.15.3:0/2182494752 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbfc8016040 con 0x55745ba4d6e0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfb7400640 1 -- 172.21.15.3:0/2182494752 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfc8037050 con 0x55745ba4d6e0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfdb096740 10 client.25675 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55745ba6f910 con 0x55745ba4d6e0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.478+0000 7fbfb7400640 1 -- 172.21.15.3:0/2182494752 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc8004cc0 con 0x55745ba4d6e0 2024-08-01T04:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfb7400640 1 -- 172.21.15.3:0/2182494752 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fbfc8028ca0 con 0x55745ba4d6e0 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfdb096740 10 client.25675 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfdb096740 10 client.25675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfdb096740 20 client.25675 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfdb096740 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745ba70d70 0x55745ba73160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfdb096740 4 client.25675 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55745b9c0160 con 0x55745ba70d70 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.479+0000 7fbfd5200640 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745ba70d70 0x55745ba73160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.480+0000 7fbfd5200640 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745ba70d70 0x55745ba73160 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ecbf3b7e4ac4e7d server_cookie=448a0218eef8b321 in_seq=0 out_seq=0 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.480+0000 7fbfb7400640 10 client.25675 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfb7400640 1 -- 172.21.15.3:0/2182494752 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55745b9c0160 con 0x55745ba70d70 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfb7400640 10 client.25675 handle_command_reply: tid=0 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 client.25675 shutdown 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 msgr2=0x55745ba55740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba55740 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8004640 tx=0x7fbfc8004540 comp rx=0 tx=0).stop 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 shutdown_connections 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745ba70d70 0x55745ba73160 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba55c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfb7400640 0 client.25675 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 --2- 172.21.15.3:0/2182494752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba55740 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 >> 172.21.15.3:0/2182494752 conn(0x55745ba49c70 msgr2=0x55745ba4f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.481+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 shutdown_connections 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.482+0000 7fbfdb096740 1 -- 172.21.15.3:0/2182494752 wait complete. 2024-08-01T04:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.482+0000 7fbfdb096740 20 client.25675 trim_cache size 0 max 16384 2024-08-01T04:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.496+0000 7fbfdb096740 1 Processor -- start 2024-08-01T04:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 -- start start 2024-08-01T04:18:14.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba44910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba78e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01adcb0 con 0x55745ba4e000 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01ab580 con 0x55745ba588c0 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfdb096740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01a93d0 con 0x55745ba4d6e0 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfd5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfd5200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba788f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60388/0 (socket says 172.21.15.3:60388) 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfd5200640 1 -- 172.21.15.3:0/3785409153 learned_addr learned my addr 172.21.15.3:0/3785409153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.497+0000 7fbfd6600640 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba44910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd6600640 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba78e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd5200640 1 -- 172.21.15.3:0/3785409153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 msgr2=0x55745ba44910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd5200640 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba44910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd5200640 1 -- 172.21.15.3:0/3785409153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 msgr2=0x55745ba78e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd5200640 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba78e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd5200640 1 -- 172.21.15.3:0/3785409153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55745ba4a900 con 0x55745ba4e000 2024-08-01T04:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd6600640 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba44910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:14.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfd5200640 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba788f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009b60 tx=0x7fbfbc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f977115d0a516de8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:14.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.498+0000 7fbfc2000640 1 -- 172.21.15.3:0/3785409153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc022070 con 0x55745ba4e000 2024-08-01T04:18:14.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfc2000640 1 -- 172.21.15.3:0/3785409153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc0047a0 con 0x55745ba4e000 2024-08-01T04:18:14.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfc2000640 1 -- 172.21.15.3:0/3785409153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc01e8e0 con 0x55745ba4e000 2024-08-01T04:18:14.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 -- 172.21.15.3:0/3785409153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 msgr2=0x55745ba788f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba788f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009b60 tx=0x7fbfbc005f90 comp rx=0 tx=0).stop 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 -- 172.21.15.3:0/3785409153 shutdown_connections 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba588c0 0x55745ba78e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba4e000 0x55745ba788f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 --2- 172.21.15.3:0/3785409153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4d6e0 0x55745ba44910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.499+0000 7fbfdb096740 1 -- 172.21.15.3:0/3785409153 >> 172.21.15.3:0/3785409153 conn(0x55745ba49c70 msgr2=0x55745ba577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.500+0000 7fbfdb096740 1 -- 172.21.15.3:0/3785409153 shutdown_connections 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.500+0000 7fbfdb096740 1 -- 172.21.15.3:0/3785409153 wait complete. 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.500+0000 7fbfdb096740 1 -- start start 2024-08-01T04:18:14.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.500+0000 7fbfdb096740 1 Processor -- start 2024-08-01T04:18:14.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba69300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba51f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01a9a40 con 0x55745ba588c0 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01aaeb0 con 0x55745ba4d6e0 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01aa110 con 0x55745ba4e000 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfdb096740 10 client.0 _ll_register_callbacks cb 0x7fbfc8004b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba51f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfd5200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba51f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60402/0 (socket says 172.21.15.3:60402) 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfd5200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba69840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:47490/0 (socket says 172.21.15.3:47490) 2024-08-01T04:18:14.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfd6600640 1 -- 172.21.15.3:0/1342883724 learned_addr learned my addr 172.21.15.3:0/1342883724 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.501+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba69300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd5200640 1 -- 172.21.15.3:0/1342883724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 msgr2=0x55745ba69300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd5200640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba69300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd5200640 1 -- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 msgr2=0x55745ba51f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd5200640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba51f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd5200640 1 -- 172.21.15.3:0/1342883724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55745ba50550 con 0x55745ba4e000 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba51f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfd5200640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba69840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55745ba56900 tx=0x7fbfbc004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec1ddb83880f90a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.502+0000 7fbfb7400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfb7400640 1 -- 172.21.15.3:0/1342883724 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc022070 con 0x55745ba4e000 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfb7400640 1 -- 172.21.15.3:0/1342883724 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbfbc004cd0 con 0x55745ba4e000 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc0047a0 con 0x55745ba4e000 2024-08-01T04:18:14.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfb7400640 1 -- 172.21.15.3:0/1342883724 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc005000 con 0x55745ba4e000 2024-08-01T04:18:14.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfdb096740 10 client.25681 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:14.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55745badb150 con 0x55745ba4e000 2024-08-01T04:18:14.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.503+0000 7fbfb7400640 1 -- 172.21.15.3:0/1342883724 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc0047a0 con 0x55745ba4e000 2024-08-01T04:18:14.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfb7400640 1 -- 172.21.15.3:0/1342883724 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fbfbc026400 con 0x55745ba4e000 2024-08-01T04:18:14.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfdb096740 10 client.25681 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:14.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfdb096740 10 client.25681 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfdb096740 20 client.25681 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745badb790 0x55745baddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfdb096740 4 client.25681 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55745b9ba670 con 0x55745badb790 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.504+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745badb790 0x55745baddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfd6600640 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745badb790 0x55745baddbb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49af535284529f87 server_cookie=277cc85344950ccb in_seq=0 out_seq=0 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfb7400640 10 client.25681 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfb7400640 1 -- 172.21.15.3:0/1342883724 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55745b9ba670 con 0x55745badb790 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfb7400640 10 client.25681 handle_command_reply: tid=0 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 client.25681 shutdown 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 msgr2=0x55745ba69840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba69840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55745ba56900 tx=0x7fbfbc004950 comp rx=0 tx=0).stop 2024-08-01T04:18:14.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 shutdown_connections 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55745badb790 0x55745baddbb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55745ba588c0 0x55745ba51f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.505+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55745ba4e000 0x55745ba69840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.506+0000 7fbfdb096740 1 --2- 172.21.15.3:0/1342883724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55745ba4d6e0 0x55745ba69300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.506+0000 7fbfb7400640 0 client.25681 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.506+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 >> 172.21.15.3:0/1342883724 conn(0x55745ba49c70 msgr2=0x55745ba6bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.506+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 shutdown_connections 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.506+0000 7fbfdb096740 1 -- 172.21.15.3:0/1342883724 wait complete. 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.506+0000 7fbfdb096740 20 client.25681 trim_cache size 0 max 16384 2024-08-01T04:18:14.509 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:18:14.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbfb80a2d70 msgr2=0x7fbfb80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbfb80a2d70 0x7fbfb80a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55745ba4aad0 tx=0x7fbfc40058b0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 msgr2=0x7fbfd019a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:14.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 0x7fbfd019a1b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006010 tx=0x7fbfc4003c10 comp rx=0 tx=0).stop 2024-08-01T04:18:14.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 shutdown_connections 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd0109900 0x7fbfd019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbfb80a2d70 0x7fbfb80a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd01042a0 0x7fbfd019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 --2- 172.21.15.3:0/2886364855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfd0103900 0x7fbfd019a1b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 >> 172.21.15.3:0/2886364855 conn(0x7fbfd00ff600 msgr2=0x7fbfd00ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 shutdown_connections 2024-08-01T04:18:14.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:14.507+0000 7fbfd7000640 1 -- 172.21.15.3:0/2886364855 wait complete. 2024-08-01T04:18:14.541 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to run the C preprocessor... gcc -E 2024-08-01T04:18:14.637 INFO:tasks.workunit.client.0.smithi003.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-08-01T04:18:14.642 INFO:tasks.workunit.client.0.smithi003.stdout:checking for egrep... /usr/bin/grep -E 2024-08-01T04:18:14.670 INFO:tasks.workunit.client.0.smithi003.stdout:checking for AIX... no 2024-08-01T04:18:14.690 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... (cached) gcc 2024-08-01T04:18:14.718 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-08-01T04:18:14.720 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... (cached) yes 2024-08-01T04:18:14.720 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-08-01T04:18:14.721 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... (cached) gcc3 2024-08-01T04:18:14.908 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lm... yes 2024-08-01T04:18:15.074 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lpthread... yes 2024-08-01T04:18:15.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:15 smithi003 ceph-mon[32262]: pgmap v62: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 112 KiB/s wr, 11 op/s 2024-08-01T04:18:15.472 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... yes 2024-08-01T04:18:15.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:15 smithi163 ceph-mon[37383]: pgmap v62: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 112 KiB/s wr, 11 op/s 2024-08-01T04:18:15.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:18:15.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:15 smithi160 ceph-mon[37224]: pgmap v62: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 112 KiB/s wr, 11 op/s 2024-08-01T04:18:15.655 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-08-01T04:18:15.753 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/types.h... yes 2024-08-01T04:18:15.869 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/stat.h... yes 2024-08-01T04:18:15.974 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdlib.h... yes 2024-08-01T04:18:16.070 INFO:tasks.workunit.client.0.smithi003.stdout:checking for string.h... yes 2024-08-01T04:18:16.077 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:16.291 INFO:tasks.workunit.client.0.smithi003.stdout:checking for memory.h... yes 2024-08-01T04:18:16.411 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strings.h... yes 2024-08-01T04:18:16.527 INFO:tasks.workunit.client.0.smithi003.stdout:checking for inttypes.h... yes 2024-08-01T04:18:16.644 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... yes 2024-08-01T04:18:16.761 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... yes 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.761+0000 7f5edaa00640 1 -- 172.21.15.3:0/1163901202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed4102a80 msgr2=0x7f5ed4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.761+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1163901202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed4102a80 0x7f5ed4102e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009900 tx=0x7f5ec402a480 comp rx=0 tx=0).stop 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 -- 172.21.15.3:0/1163901202 shutdown_connections 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1163901202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed410beb0 0x7f5ed410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1163901202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed41033a0 0x7f5ed410b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1163901202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed4102a80 0x7f5ed4102e60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 -- 172.21.15.3:0/1163901202 >> 172.21.15.3:0/1163901202 conn(0x7f5ed40fc500 msgr2=0x7f5ed40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 -- 172.21.15.3:0/1163901202 shutdown_connections 2024-08-01T04:18:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.762+0000 7f5edaa00640 1 -- 172.21.15.3:0/1163901202 wait complete. 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 Processor -- start 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 -- start start 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 0x7f5ed419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed41033a0 0x7f5ed419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed410beb0 0x7f5ed419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4111d00 con 0x7f5ed4102a80 2024-08-01T04:18:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4111b80 con 0x7f5ed410beb0 2024-08-01T04:18:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.763+0000 7f5edaa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4111e80 con 0x7f5ed41033a0 2024-08-01T04:18:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 0x7f5ed419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed41033a0 0x7f5ed419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 0x7f5ed419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60416/0 (socket says 172.21.15.3:60416) 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 -- 172.21.15.3:0/1970374794 learned_addr learned my addr 172.21.15.3:0/1970374794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed410beb0 0x7f5ed419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 -- 172.21.15.3:0/1970374794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed41033a0 msgr2=0x7f5ed419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed41033a0 0x7f5ed419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 -- 172.21.15.3:0/1970374794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed410beb0 msgr2=0x7f5ed419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed410beb0 0x7f5ed419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3e00640 1 -- 172.21.15.3:0/1970374794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed419f350 con 0x7f5ed4102a80 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.764+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed41033a0 0x7f5ed419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:16.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.765+0000 7f5ed3e00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 0x7f5ed419a3a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4004770 tx=0x7f5ec4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.765+0000 7f5ed0c00640 1 -- 172.21.15.3:0/1970374794 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec403f070 con 0x7f5ed4102a80 2024-08-01T04:18:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.765+0000 7f5ed0c00640 1 -- 172.21.15.3:0/1970374794 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4031ca0 con 0x7f5ed4102a80 2024-08-01T04:18:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.765+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed419f5e0 con 0x7f5ed4102a80 2024-08-01T04:18:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.765+0000 7f5ed0c00640 1 -- 172.21.15.3:0/1970374794 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4035030 con 0x7f5ed4102a80 2024-08-01T04:18:16.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.765+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a6f20 con 0x7f5ed4102a80 2024-08-01T04:18:16.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.769+0000 7f5ed0c00640 1 -- 172.21.15.3:0/1970374794 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5ec4042070 con 0x7f5ed4102a80 2024-08-01T04:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.769+0000 7f5edd3d3740 1 Processor -- start 2024-08-01T04:18:16.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 -- start start 2024-08-01T04:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e58c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56258e4e6bf0 con 0x56258e5814f0 2024-08-01T04:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56258e47eb20 con 0x56258e58c6d0 2024-08-01T04:18:16.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56258e4e2bb0 con 0x56258e581e10 2024-08-01T04:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.770+0000 7f5ed8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5818d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60432/0 (socket says 172.21.15.3:60432) 2024-08-01T04:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 -- 172.21.15.3:0/3769028971 learned_addr learned my addr 172.21.15.3:0/3769028971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed3e00640 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e58c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed3400640 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 -- 172.21.15.3:0/3769028971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 msgr2=0x56258e58c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e58c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 -- 172.21.15.3:0/3769028971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 msgr2=0x56258e58eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed8a00640 1 -- 172.21.15.3:0/3769028971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258e4857f0 con 0x56258e5814f0 2024-08-01T04:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed3400640 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:16.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.771+0000 7f5ed0c00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5eb80a2d70 0x7f5eb80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.772+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5818d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009f50 tx=0x7f5ecc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd5969fabd22cfa server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.772+0000 7f5ed0c00640 1 -- 172.21.15.3:0/1970374794 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f5ec40efa00 con 0x7f5ed4102a80 2024-08-01T04:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.772+0000 7f5ed3e00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5eb80a2d70 0x7f5eb80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.772+0000 7f5ec9200640 1 -- 172.21.15.3:0/3769028971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc004d20 con 0x56258e5814f0 2024-08-01T04:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.772+0000 7f5ec9200640 1 -- 172.21.15.3:0/3769028971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ecc004050 con 0x56258e5814f0 2024-08-01T04:18:16.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.772+0000 7f5ec9200640 1 -- 172.21.15.3:0/3769028971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc005280 con 0x56258e5814f0 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 -- 172.21.15.3:0/3769028971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 msgr2=0x56258e5818d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5818d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009f50 tx=0x7f5ecc0118d0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5ed3e00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5eb80a2d70 0x7f5eb80a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x56258e57e8e0 tx=0x7f5ec40f5510 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 -- 172.21.15.3:0/3769028971 shutdown_connections 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e58c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/3769028971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5818d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 -- 172.21.15.3:0/3769028971 >> 172.21.15.3:0/3769028971 conn(0x56258e57da80 msgr2=0x56258e57de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.773+0000 7f5edd3d3740 1 -- 172.21.15.3:0/3769028971 shutdown_connections 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.774+0000 7f5edd3d3740 1 -- 172.21.15.3:0/3769028971 wait complete. 2024-08-01T04:18:16.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.774+0000 7f5edd3d3740 1 -- start start 2024-08-01T04:18:16.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.774+0000 7f5edd3d3740 1 Processor -- start 2024-08-01T04:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e5814f0 0x56258e5895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e589b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e58c6d0 0x56258e58a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56258e58f2a0 con 0x56258e58c6d0 2024-08-01T04:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56258e4a1cf0 con 0x56258e581e10 2024-08-01T04:18:16.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56258e58f420 con 0x56258e5814f0 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5edd3d3740 10 client.0 _ll_register_callbacks cb 0x56258e48de30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e589b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e589b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52606/0 (socket says 172.21.15.3:52606) 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 -- 172.21.15.3:0/1515333814 learned_addr learned my addr 172.21.15.3:0/1515333814 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e5814f0 0x56258e5895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e58c6d0 0x56258e58a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 -- 172.21.15.3:0/1515333814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e5814f0 msgr2=0x56258e5895d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e5814f0 0x56258e5895d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 -- 172.21.15.3:0/1515333814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e58c6d0 msgr2=0x56258e58a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e58c6d0 0x56258e58a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed3400640 1 -- 172.21.15.3:0/1515333814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258e2583c0 con 0x56258e581e10 2024-08-01T04:18:16.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.775+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e58c6d0 0x56258e58a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e589b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0012d30 tx=0x7f5ec000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efb12a88f21584a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5ebd600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5ebd600640 1 -- 172.21.15.3:0/1515333814 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000fe20 con 0x56258e581e10 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5ebd600640 1 -- 172.21.15.3:0/1515333814 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f5ec00120d0 con 0x56258e581e10 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ecc004050 con 0x56258e581e10 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5ebd600640 1 -- 172.21.15.3:0/1515333814 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ec0010430 con 0x56258e581e10 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5edd3d3740 10 client.25703 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.776+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56258e59aff0 con 0x56258e581e10 2024-08-01T04:18:16.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.777+0000 7f5ebd600640 1 -- 172.21.15.3:0/1515333814 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ecc004050 con 0x56258e581e10 2024-08-01T04:18:16.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5ebd600640 1 -- 172.21.15.3:0/1515333814 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5ec0022030 con 0x56258e581e10 2024-08-01T04:18:16.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5edd3d3740 10 client.25703 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5edd3d3740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5edd3d3740 20 client.25703 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e59c500 0x56258e59e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5edd3d3740 4 client.25703 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258e4f4160 con 0x56258e59c500 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e59c500 0x56258e59e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e59c500 0x56258e59e8f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a710c64a6053e7f2 server_cookie=dc48e065f7e881ee in_seq=0 out_seq=0 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.778+0000 7f5ebd600640 10 client.25703 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.779+0000 7f5ebd600640 1 -- 172.21.15.3:0/1515333814 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56258e4f4160 con 0x56258e59c500 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.779+0000 7f5ebd600640 10 client.25703 handle_command_reply: tid=0 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.779+0000 7f5edd3d3740 1 client.25703 shutdown 2024-08-01T04:18:16.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 msgr2=0x56258e589b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e589b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0012d30 tx=0x7f5ec000f790 comp rx=0 tx=0).stop 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 shutdown_connections 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e59c500 0x56258e59e8f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e58c6d0 0x56258e58a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e589b10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1515333814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e5814f0 0x56258e5895d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 >> 172.21.15.3:0/1515333814 conn(0x56258e57da80 msgr2=0x56258e57cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5ebd600640 0 client.25703 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 shutdown_connections 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1515333814 wait complete. 2024-08-01T04:18:16.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.780+0000 7f5edd3d3740 20 client.25703 trim_cache size 0 max 16384 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 Processor -- start 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 -- start start 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e5a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e58c6d0 0x56258e5a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41b2220 con 0x56258e5814f0 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41afaf0 con 0x56258e581e10 2024-08-01T04:18:16.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.796+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41ad940 con 0x56258e58c6d0 2024-08-01T04:18:16.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e5a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e5a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52616/0 (socket says 172.21.15.3:52616) 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 -- 172.21.15.3:0/1104565743 learned_addr learned my addr 172.21.15.3:0/1104565743 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e58c6d0 0x56258e5a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 -- 172.21.15.3:0/1104565743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e58c6d0 msgr2=0x56258e5a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e58c6d0 0x56258e5a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 -- 172.21.15.3:0/1104565743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 msgr2=0x56258e5849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5849b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed3400640 1 -- 172.21.15.3:0/1104565743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258e581320 con 0x56258e581e10 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.797+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.798+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e5a41c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0012370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2112cf0daf6467a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.798+0000 7f5ebfe00640 1 -- 172.21.15.3:0/1104565743 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00120d0 con 0x56258e581e10 2024-08-01T04:18:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.798+0000 7f5ebfe00640 1 -- 172.21.15.3:0/1104565743 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0005520 con 0x56258e581e10 2024-08-01T04:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.798+0000 7f5ebfe00640 1 -- 172.21.15.3:0/1104565743 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001ca70 con 0x56258e581e10 2024-08-01T04:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1104565743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 msgr2=0x56258e5a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e5a41c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0012370 comp rx=0 tx=0).stop 2024-08-01T04:18:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1104565743 shutdown_connections 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e58c6d0 0x56258e5a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e581e10 0x56258e5a41c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1104565743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e5849b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1104565743 >> 172.21.15.3:0/1104565743 conn(0x56258e57da80 msgr2=0x56258e5886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1104565743 shutdown_connections 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1104565743 wait complete. 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.799+0000 7f5edd3d3740 1 -- start start 2024-08-01T04:18:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 Processor -- start 2024-08-01T04:18:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e594970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e594eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41adfb0 con 0x56258e5814f0 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41af420 con 0x56258e58c6d0 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41ae680 con 0x56258e581e10 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.800+0000 7f5edd3d3740 10 client.0 _ll_register_callbacks cb 0x7f5ec0012910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e594970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e594970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60468/0 (socket says 172.21.15.3:60468) 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 -- 172.21.15.3:0/1746876435 learned_addr learned my addr 172.21.15.3:0/1746876435 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e594eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 -- 172.21.15.3:0/1746876435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 msgr2=0x56258e594eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e594eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 -- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 msgr2=0x56258e58fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed8a00640 1 -- 172.21.15.3:0/1746876435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258e590530 con 0x56258e5814f0 2024-08-01T04:18:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.801+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e594eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5ed8a00640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e594970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc015040 tx=0x7f5ecc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79aed20ba22e601 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5ebd600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5ebd600640 1 -- 172.21.15.3:0/1746876435 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc004820 con 0x56258e5814f0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec0005520 con 0x56258e5814f0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5ebd600640 1 -- 172.21.15.3:0/1746876435 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f5ecc0049c0 con 0x56258e5814f0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5ebd600640 1 -- 172.21.15.3:0/1746876435 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ecc004bc0 con 0x56258e5814f0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5edd3d3740 10 client.15564 fetch_fsmap learned FSMap version 26 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.802+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56258e606d10 con 0x56258e5814f0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5ebd600640 1 -- 172.21.15.3:0/1746876435 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0005520 con 0x56258e5814f0 2024-08-01T04:18:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5ebd600640 1 -- 172.21.15.3:0/1746876435 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5ecc03f030 con 0x56258e5814f0 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5edd3d3740 10 client.15564 fetch_fsmap finished waiting for FSMap version 26 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5edd3d3740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5edd3d3740 20 client.15564 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e607290 0x56258e609680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5edd3d3740 4 client.15564 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56258e4ee670 con 0x56258e607290 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.803+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e607290 0x56258e609680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.804+0000 7f5ed3400640 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e607290 0x56258e609680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1298ff082abb250a server_cookie=237de0abac5b5bc7 in_seq=0 out_seq=0 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.804+0000 7f5ebd600640 10 client.15564 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.804+0000 7f5ebd600640 1 -- 172.21.15.3:0/1746876435 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56258e4ee670 con 0x56258e607290 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.804+0000 7f5ebd600640 10 client.15564 handle_command_reply: tid=0 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.804+0000 7f5edd3d3740 1 client.15564 shutdown 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 msgr2=0x56258e594970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e594970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc015040 tx=0x7f5ecc005fd0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 shutdown_connections 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56258e607290 0x56258e609680 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56258e58c6d0 0x56258e58fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56258e581e10 0x56258e594eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 --2- 172.21.15.3:0/1746876435 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56258e5814f0 0x56258e594970 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 >> 172.21.15.3:0/1746876435 conn(0x56258e57da80 msgr2=0x56258e589590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5ebd600640 0 client.15564 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 shutdown_connections 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 1 -- 172.21.15.3:0/1746876435 wait complete. 2024-08-01T04:18:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.805+0000 7f5edd3d3740 20 client.15564 trim_cache size 0 max 16384 2024-08-01T04:18:16.809 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:18:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5eb80a2d70 msgr2=0x7f5eb80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5eb80a2d70 0x7f5eb80a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x56258e57e8e0 tx=0x7f5ec40f5510 comp rx=0 tx=0).stop 2024-08-01T04:18:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 msgr2=0x7f5ed419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 0x7f5ed419a3a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4004770 tx=0x7f5ec4003c10 comp rx=0 tx=0).stop 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 shutdown_connections 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ed410beb0 0x7f5ed419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5eb80a2d70 0x7f5eb80a5230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed41033a0 0x7f5ed419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 --2- 172.21.15.3:0/1970374794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ed4102a80 0x7f5ed419a3a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 >> 172.21.15.3:0/1970374794 conn(0x7f5ed40fc500 msgr2=0x7f5ed410d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 shutdown_connections 2024-08-01T04:18:16.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:16.806+0000 7f5edaa00640 1 -- 172.21.15.3:0/1970374794 wait complete. 2024-08-01T04:18:16.941 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h usability... yes 2024-08-01T04:18:16.994 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h presence... yes 2024-08-01T04:18:16.997 INFO:tasks.workunit.client.0.smithi003.stdout:checking for pthread.h... yes 2024-08-01T04:18:17.120 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h usability... yes 2024-08-01T04:18:17.172 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h presence... yes 2024-08-01T04:18:17.173 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fcntl.h... yes 2024-08-01T04:18:17.293 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h usability... yes 2024-08-01T04:18:17.339 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h presence... yes 2024-08-01T04:18:17.341 INFO:tasks.workunit.client.0.smithi003.stdout:checking for limits.h... yes 2024-08-01T04:18:17.378 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... (cached) yes 2024-08-01T04:18:17.492 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h usability... yes 2024-08-01T04:18:17.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:17 smithi163 ceph-mon[37383]: pgmap v63: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 112 KiB/s wr, 11 op/s 2024-08-01T04:18:17.547 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h presence... yes 2024-08-01T04:18:17.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:17 smithi003 ceph-mon[32262]: pgmap v63: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 112 KiB/s wr, 11 op/s 2024-08-01T04:18:17.549 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/time.h... yes 2024-08-01T04:18:17.586 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... (cached) yes 2024-08-01T04:18:17.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:17 smithi160 ceph-mon[37224]: pgmap v63: 97 pgs: 97 active+clean; 2.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 112 KiB/s wr, 11 op/s 2024-08-01T04:18:17.707 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h usability... yes 2024-08-01T04:18:17.761 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h presence... yes 2024-08-01T04:18:17.762 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/vfs.h... yes 2024-08-01T04:18:17.879 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h usability... no 2024-08-01T04:18:17.942 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h presence... no 2024-08-01T04:18:17.944 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/limits.h... no 2024-08-01T04:18:18.036 INFO:tasks.workunit.client.0.smithi003.stdout:checking for an ANSI C-conforming const... yes 2024-08-01T04:18:18.130 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-01T04:18:18.133 INFO:tasks.workunit.client.0.smithi003.stdout:checking for function prototypes... yes 2024-08-01T04:18:18.214 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether setvbuf arguments are reversed... no 2024-08-01T04:18:18.374 INFO:tasks.workunit.client.0.smithi003.stdout:checking for system... yes 2024-08-01T04:18:18.525 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gettimeofday... yes 2024-08-01T04:18:18.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:18 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:18:18.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:18 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:18:18.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:18 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:18:18.666 INFO:tasks.workunit.client.0.smithi003.stdout:checking for mkdir... yes 2024-08-01T04:18:18.754 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:18:18.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:18:18.811 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strerror... yes 2024-08-01T04:18:18.942 INFO:tasks.workunit.client.0.smithi003.stdout:checking for open64... yes 2024-08-01T04:18:19.076 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stat64... yes 2024-08-01T04:18:19.202 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fseeko64... yes 2024-08-01T04:18:19.217 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:19.329 INFO:tasks.workunit.client.0.smithi003.stdout:checking for lrand48_r... yes 2024-08-01T04:18:19.469 INFO:tasks.workunit.client.0.smithi003.stdout:checking for srand48_r... yes 2024-08-01T04:18:19.523 INFO:tasks.workunit.client.0.smithi003.stdout:configure: creating ./config.status 2024-08-01T04:18:19.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:19 smithi163 ceph-mon[37383]: pgmap v64: 97 pgs: 97 active+clean; 4.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 320 KiB/s wr, 41 op/s 2024-08-01T04:18:19.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:19 smithi003 ceph-mon[32262]: pgmap v64: 97 pgs: 97 active+clean; 4.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 320 KiB/s wr, 41 op/s 2024-08-01T04:18:19.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:19 smithi160 ceph-mon[37224]: pgmap v64: 97 pgs: 97 active+clean; 4.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 320 KiB/s wr, 41 op/s 2024-08-01T04:18:19.793 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating Makefile 2024-08-01T04:18:19.835 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating config.h 2024-08-01T04:18:19.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 -- 172.21.15.3:0/106841527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8103ae0 msgr2=0x7f31d8103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 --2- 172.21.15.3:0/106841527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8103ae0 0x7f31d8103f40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009930 tx=0x7f31cc02a420 comp rx=0 tx=0).stop 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 -- 172.21.15.3:0/106841527 shutdown_connections 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 --2- 172.21.15.3:0/106841527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8104480 0x7f31d81109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 --2- 172.21.15.3:0/106841527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8103ae0 0x7f31d8103f40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 --2- 172.21.15.3:0/106841527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31d8109b70 0x7f31d8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 -- 172.21.15.3:0/106841527 >> 172.21.15.3:0/106841527 conn(0x7f31d80ff780 msgr2=0x7f31d8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 -- 172.21.15.3:0/106841527 shutdown_connections 2024-08-01T04:18:19.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.877+0000 7f31df600640 1 -- 172.21.15.3:0/106841527 wait complete. 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.878+0000 7f31df600640 1 Processor -- start 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 -- start start 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 0x7f31d819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31d8104480 0x7f31d819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8109b70 0x7f31d819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8113f30 con 0x7f31d8109b70 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8113db0 con 0x7f31d8103ae0 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31df600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81140b0 con 0x7f31d8104480 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 0x7f31d819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 0x7f31d819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52628/0 (socket says 172.21.15.3:52628) 2024-08-01T04:18:19.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31dcc00640 1 -- 172.21.15.3:0/1930614461 learned_addr learned my addr 172.21.15.3:0/1930614461 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.879+0000 7f31dd600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8109b70 0x7f31d819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31d7e00640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31d8104480 0x7f31d819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dcc00640 1 -- 172.21.15.3:0/1930614461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31d8104480 msgr2=0x7f31d819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31d8104480 0x7f31d819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dcc00640 1 -- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8109b70 msgr2=0x7f31d819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8109b70 0x7f31d819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dcc00640 1 -- 172.21.15.3:0/1930614461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d819f3c0 con 0x7f31d8103ae0 2024-08-01T04:18:19.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dd600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8109b70 0x7f31d819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.880+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 0x7f31d819a410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31c800e9f0 tx=0x7f31c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.881+0000 7f31d5600640 1 -- 172.21.15.3:0/1930614461 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c800cd80 con 0x7f31d8103ae0 2024-08-01T04:18:19.883 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing depfiles commands 2024-08-01T04:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.881+0000 7f31d5600640 1 -- 172.21.15.3:0/1930614461 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c8005500 con 0x7f31d8103ae0 2024-08-01T04:18:19.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.881+0000 7f31d5600640 1 -- 172.21.15.3:0/1930614461 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8010690 con 0x7f31d8103ae0 2024-08-01T04:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.881+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d819f6b0 con 0x7f31d8103ae0 2024-08-01T04:18:19.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.881+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d81a6f90 con 0x7f31d8103ae0 2024-08-01T04:18:19.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.884+0000 7f31e1fdc740 1 Processor -- start 2024-08-01T04:18:19.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.884+0000 7f31e1fdc740 1 -- start start 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31d5600640 1 -- 172.21.15.3:0/1930614461 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f31c80046c0 con 0x7f31d8103ae0 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab8985860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab8990120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab8992a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564ab88a07e0 con 0x564ab8985480 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x564ab88ebac0 con 0x564ab8985da0 2024-08-01T04:18:19.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564ab88838c0 con 0x564ab8990660 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31dd600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab8985860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab8990120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31dd600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab8985860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60494/0 (socket says 172.21.15.3:60494) 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.885+0000 7f31dd600640 1 -- 172.21.15.3:0/1743051825 learned_addr learned my addr 172.21.15.3:0/1743051825 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.886+0000 7f31dcc00640 1 -- 172.21.15.3:0/1743051825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 msgr2=0x564ab8992a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.886+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab8992a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.886+0000 7f31dcc00640 1 -- 172.21.15.3:0/1743051825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 msgr2=0x564ab8985860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.886+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab8985860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.886+0000 7f31dcc00640 1 -- 172.21.15.3:0/1743051825 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab888a760 con 0x564ab8985da0 2024-08-01T04:18:19.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.887+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab8990120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31c80ccf50 tx=0x7f31c80d5b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c49e1b47bdafa58 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:19.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.887+0000 7f31c5c00640 1 -- 172.21.15.3:0/1743051825 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80de070 con 0x564ab8985da0 2024-08-01T04:18:19.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.887+0000 7f31c5c00640 1 -- 172.21.15.3:0/1743051825 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c80d10d0 con 0x564ab8985da0 2024-08-01T04:18:19.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.887+0000 7f31c5c00640 1 -- 172.21.15.3:0/1743051825 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80da660 con 0x564ab8985da0 2024-08-01T04:18:19.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.887+0000 7f31d5600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f31c00a2d70 0x7f31c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31d5600640 1 -- 172.21.15.3:0/1930614461 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f31c80cf8a0 con 0x7f31d8103ae0 2024-08-01T04:18:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31dd600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f31c00a2d70 0x7f31c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1743051825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 msgr2=0x564ab8990120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab8990120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f31c80ccf50 tx=0x7f31c80d5b20 comp rx=0 tx=0).stop 2024-08-01T04:18:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1743051825 shutdown_connections 2024-08-01T04:18:19.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab8992a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab8990120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1743051825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab8985860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.888+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1743051825 >> 172.21.15.3:0/1743051825 conn(0x564ab8981a10 msgr2=0x564ab8981df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:19.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.889+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1743051825 shutdown_connections 2024-08-01T04:18:19.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.889+0000 7f31dd600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f31c00a2d70 0x7f31c00a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31d00099a0 tx=0x7f31d0009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:19.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.889+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1743051825 wait complete. 2024-08-01T04:18:19.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 -- start start 2024-08-01T04:18:19.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 Processor -- start 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 0x564ab898d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985da0 0x564ab898dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab898e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564ab88a6ae0 con 0x564ab8985da0 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x564ab88e7a80 con 0x564ab8985480 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.890+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564ab89930f0 con 0x564ab8990660 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31e1fdc740 10 client.0 _ll_register_callbacks cb 0x564ab88a8190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 0x564ab898d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985da0 0x564ab898dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 0x564ab898d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52648/0 (socket says 172.21.15.3:52648) 2024-08-01T04:18:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 -- 172.21.15.3:0/640237340 learned_addr learned my addr 172.21.15.3:0/640237340 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:19.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dd600640 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab898e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 -- 172.21.15.3:0/640237340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 msgr2=0x564ab898e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab898e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 -- 172.21.15.3:0/640237340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985da0 msgr2=0x564ab898dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985da0 0x564ab898dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.891+0000 7f31dcc00640 1 -- 172.21.15.3:0/640237340 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab865d6a0 con 0x564ab8985480 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31dcc00640 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 0x564ab898d670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31c809bcb0 tx=0x7f31c8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205c2497015c7d24 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:19.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31ba000640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31ba000640 1 -- 172.21.15.3:0/640237340 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80de030 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31ba000640 1 -- 172.21.15.3:0/640237340 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f31c80cd1d0 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c80d10d0 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31ba000640 1 -- 172.21.15.3:0/640237340 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31c80ba070 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31e1fdc740 10 client.25721 fetch_fsmap learned FSMap version 27 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ab899ee50 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.892+0000 7f31ba000640 1 -- 172.21.15.3:0/640237340 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c80d10d0 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31ba000640 1 -- 172.21.15.3:0/640237340 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f31c809bf80 con 0x564ab8985480 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31e1fdc740 10 client.25721 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31e1fdc740 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31e1fdc740 20 client.25721 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab89a01a0 0x564ab89a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31e1fdc740 4 client.25721 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab88f9030 con 0x564ab89a01a0 2024-08-01T04:18:19.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.893+0000 7f31d7e00640 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab89a01a0 0x564ab89a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.894+0000 7f31d7e00640 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab89a01a0 0x564ab89a2590 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d78ea3264cdf72b server_cookie=ec82e9e35b8a47c in_seq=0 out_seq=0 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.894+0000 7f31ba000640 10 client.25721 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31ba000640 1 -- 172.21.15.3:0/640237340 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564ab88f9030 con 0x564ab89a01a0 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31ba000640 10 client.25721 handle_command_reply: tid=0 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 client.25721 shutdown 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 msgr2=0x564ab898d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 0x564ab898d670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31c809bcb0 tx=0x7f31c8009310 comp rx=0 tx=0).stop 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 shutdown_connections 2024-08-01T04:18:19.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab89a01a0 0x564ab89a2590 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8990660 0x564ab898e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985da0 0x564ab898dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/640237340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985480 0x564ab898d670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 >> 172.21.15.3:0/640237340 conn(0x564ab8981a10 msgr2=0x564ab8987740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31ba000640 0 client.25721 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 shutdown_connections 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.895+0000 7f31e1fdc740 1 -- 172.21.15.3:0/640237340 wait complete. 2024-08-01T04:18:19.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.896+0000 7f31e1fdc740 20 client.25721 trim_cache size 0 max 16384 2024-08-01T04:18:19.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.915+0000 7f31e1fdc740 1 Processor -- start 2024-08-01T04:18:19.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.915+0000 7f31e1fdc740 1 -- start start 2024-08-01T04:18:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985480 0x564ab899bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab89a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8990660 0x564ab89a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81b2350 con 0x564ab8990660 2024-08-01T04:18:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81afc20 con 0x564ab8985da0 2024-08-01T04:18:19.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81ada70 con 0x564ab8985480 2024-08-01T04:18:19.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab89a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab89a51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52654/0 (socket says 172.21.15.3:52654) 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31d7e00640 1 -- 172.21.15.3:0/2181709854 learned_addr learned my addr 172.21.15.3:0/2181709854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31dd600640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8990660 0x564ab89a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31dcc00640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985480 0x564ab899bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.916+0000 7f31d7e00640 1 -- 172.21.15.3:0/2181709854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985480 msgr2=0x564ab899bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31d7e00640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985480 0x564ab899bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31d7e00640 1 -- 172.21.15.3:0/2181709854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8990660 msgr2=0x564ab89a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31d7e00640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8990660 0x564ab89a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31d7e00640 1 -- 172.21.15.3:0/2181709854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab8985200 con 0x564ab8985da0 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31dd600640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8990660 0x564ab89a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31dcc00640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985480 0x564ab899bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:19.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31d7e00640 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab89a51d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02a9e0 tx=0x7f31cc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a7d838862cc511 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31c5200640 1 -- 172.21.15.3:0/2181709854 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc04b070 con 0x564ab8985da0 2024-08-01T04:18:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31c5200640 1 -- 172.21.15.3:0/2181709854 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc005520 con 0x564ab8985da0 2024-08-01T04:18:19.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.917+0000 7f31c5200640 1 -- 172.21.15.3:0/2181709854 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc035040 con 0x564ab8985da0 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 -- 172.21.15.3:0/2181709854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 msgr2=0x564ab89a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab89a51d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02a9e0 tx=0x7f31cc000bc0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 -- 172.21.15.3:0/2181709854 shutdown_connections 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8990660 0x564ab89a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8985da0 0x564ab89a51d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/2181709854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985480 0x564ab899bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.918+0000 7f31e1fdc740 1 -- 172.21.15.3:0/2181709854 >> 172.21.15.3:0/2181709854 conn(0x564ab8981a10 msgr2=0x564ab899b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.919+0000 7f31e1fdc740 1 -- 172.21.15.3:0/2181709854 shutdown_connections 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.919+0000 7f31e1fdc740 1 -- 172.21.15.3:0/2181709854 wait complete. 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.919+0000 7f31e1fdc740 1 -- start start 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.919+0000 7f31e1fdc740 1 Processor -- start 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab89a3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 0x564ab89a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8990660 0x564ab8a09ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81ae0e0 con 0x564ab8985480 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81af550 con 0x564ab8990660 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81ae7b0 con 0x564ab8985da0 2024-08-01T04:18:19.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.920+0000 7f31e1fdc740 10 client.0 _ll_register_callbacks cb 0x7f31d0009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:19.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 0x564ab89a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31dcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab89a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 0x564ab89a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:47592/0 (socket says 172.21.15.3:47592) 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 -- 172.21.15.3:0/1790290695 learned_addr learned my addr 172.21.15.3:0/1790290695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31dd600640 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8990660 0x564ab8a09ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 -- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8990660 msgr2=0x564ab8a09ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8990660 0x564ab8a09ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 -- 172.21.15.3:0/1790290695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 msgr2=0x564ab89a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab89a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 -- 172.21.15.3:0/1790290695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab89a4050 con 0x564ab8985da0 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31d7e00640 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 0x564ab89a37c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02a940 tx=0x7f31cc005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6154a9837538b4f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:19.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.921+0000 7f31ba000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.922+0000 7f31ba000640 1 -- 172.21.15.3:0/1790290695 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc04b070 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.922+0000 7f31ba000640 1 -- 172.21.15.3:0/1790290695 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f31cc0048d0 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.922+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31cc005520 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.922+0000 7f31ba000640 1 -- 172.21.15.3:0/1790290695 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f31cc035040 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.922+0000 7f31e1fdc740 10 client.25714 fetch_fsmap learned FSMap version 27 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.922+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ab8a0b130 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31ba000640 1 -- 172.21.15.3:0/1790290695 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31cc005520 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31ba000640 1 -- 172.21.15.3:0/1790290695 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f31cc0319a0 con 0x564ab8985da0 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31e1fdc740 10 client.25714 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31e1fdc740 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:19.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31e1fdc740 20 client.25714 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab8a0b890 0x564ab8a0dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31e1fdc740 4 client.25714 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:18:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.923+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ab88f3540 con 0x564ab8a0b890 2024-08-01T04:18:19.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.924+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab8a0b890 0x564ab8a0dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.924+0000 7f31dcc00640 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab8a0b890 0x564ab8a0dcb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e8f6a6cb91890f server_cookie=d57adb7393d2f55a in_seq=0 out_seq=0 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.924+0000 7f31ba000640 10 client.25714 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.925+0000 7f31ba000640 1 -- 172.21.15.3:0/1790290695 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ab88f3540 con 0x564ab8a0b890 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.925+0000 7f31ba000640 10 client.25714 handle_command_reply: tid=0 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.925+0000 7f31e1fdc740 1 client.25714 shutdown 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 msgr2=0x564ab89a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 0x564ab89a37c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31cc02a940 tx=0x7f31cc005760 comp rx=0 tx=0).stop 2024-08-01T04:18:19.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 shutdown_connections 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x564ab8a0b890 0x564ab8a0dcb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564ab8990660 0x564ab8a09ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ab8985da0 0x564ab89a37c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 --2- 172.21.15.3:0/1790290695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564ab8985480 0x564ab89a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 >> 172.21.15.3:0/1790290695 conn(0x564ab8981a10 msgr2=0x564ab899bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31ba000640 0 client.25714 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 shutdown_connections 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 1 -- 172.21.15.3:0/1790290695 wait complete. 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.926+0000 7f31e1fdc740 20 client.25714 trim_cache size 0 max 16384 2024-08-01T04:18:19.929 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "0e13d647-c8c5-4399-9d5c-b0ff32a860fd", 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.927+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f31c00a2d70 msgr2=0x7f31c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.927+0000 7f31df600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f31c00a2d70 0x7f31c00a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31d00099a0 tx=0x7f31d0009500 comp rx=0 tx=0).stop 2024-08-01T04:18:19.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.927+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 msgr2=0x7f31d819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.927+0000 7f31df600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 0x7f31d819a410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31c800e9f0 tx=0x7f31c800edf0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 shutdown_connections 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31d8109b70 0x7f31d819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f31c00a2d70 0x7f31c00a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31d8104480 0x7f31d819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 --2- 172.21.15.3:0/1930614461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31d8103ae0 0x7f31d819a410 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 >> 172.21.15.3:0/1930614461 conn(0x7f31d80ff780 msgr2=0x7f31d80fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 shutdown_connections 2024-08-01T04:18:19.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:19.928+0000 7f31df600640 1 -- 172.21.15.3:0/1930614461 wait complete. 2024-08-01T04:18:19.995 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing default commands 2024-08-01T04:18:20.080 INFO:tasks.workunit.client.0.smithi003.stderr:+ make 2024-08-01T04:18:20.091 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-08-01T04:18:20.096 INFO:tasks.workunit.client.0.smithi003.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-08-01T04:18:20.099 INFO:tasks.workunit.client.0.smithi003.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-08-01T04:18:20.099 INFO:tasks.workunit.client.0.smithi003.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-08-01T04:18:20.099 INFO:tasks.workunit.client.0.smithi003.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-08-01T04:18:20.099 INFO:tasks.workunit.client.0.smithi003.stderr: any GNU archive site. 2024-08-01T04:18:20.107 INFO:tasks.workunit.client.0.smithi003.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-08-01T04:18:20.111 INFO:tasks.workunit.client.0.smithi003.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-08-01T04:18:20.113 INFO:tasks.workunit.client.0.smithi003.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-08-01T04:18:20.113 INFO:tasks.workunit.client.0.smithi003.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-08-01T04:18:20.113 INFO:tasks.workunit.client.0.smithi003.stderr: You might want to install the `Automake' and `Perl' packages. 2024-08-01T04:18:20.113 INFO:tasks.workunit.client.0.smithi003.stderr: Grab them from any GNU archive site. 2024-08-01T04:18:20.201 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-08-01T04:18:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:20 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:18:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:20 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:20 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:18:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:20 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:20 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:18:20.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:20 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:20 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:18:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:20 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:20 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:18:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:20 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:20 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:18:20.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:20 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:20 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:18:20.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:20 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:20 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:18:20.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:20 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:20.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:20 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:18:20.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:20 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.066 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:18:21.268 INFO:tasks.workunit.client.0.smithi003.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-08-01T04:18:21.268 INFO:tasks.workunit.client.0.smithi003.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-08-01T04:18:21.268 INFO:tasks.workunit.client.0.smithi003.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-08-01T04:18:21.268 INFO:tasks.workunit.client.0.smithi003.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-08-01T04:18:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: pgmap v65: 97 pgs: 97 active+clean; 4.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 320 KiB/s wr, 38 op/s 2024-08-01T04:18:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:18:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:18:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:18:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:21 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.462 INFO:tasks.workunit.client.0.smithi003.stdout:/bin/sh ./config.status --recheck 2024-08-01T04:18:21.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: pgmap v65: 97 pgs: 97 active+clean; 4.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 320 KiB/s wr, 38 op/s 2024-08-01T04:18:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:18:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:18:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:18:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:21 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.580 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:21.594 INFO:tasks.workunit.client.0.smithi003.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-08-01T04:18:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: pgmap v65: 97 pgs: 97 active+clean; 4.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 320 KiB/s wr, 38 op/s 2024-08-01T04:18:21.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:18:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:18:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:18:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:21 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:18:21.827 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.249+0000 7fb432200640 1 -- 172.21.15.3:0/4173245977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c102090 msgr2=0x7fb42c10e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.249+0000 7fb432200640 1 --2- 172.21.15.3:0/4173245977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c102090 0x7fb42c10e5c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb424009cd0 tx=0x7fb42402a8e0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 -- 172.21.15.3:0/4173245977 shutdown_connections 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 --2- 172.21.15.3:0/4173245977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c102090 0x7fb42c10e5c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 --2- 172.21.15.3:0/4173245977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb42c1016f0 0x7fb42c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 --2- 172.21.15.3:0/4173245977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c1076f0 0x7fb42c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 -- 172.21.15.3:0/4173245977 >> 172.21.15.3:0/4173245977 conn(0x7fb42c0fd410 msgr2=0x7fb42c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 -- 172.21.15.3:0/4173245977 shutdown_connections 2024-08-01T04:18:22.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.250+0000 7fb432200640 1 -- 172.21.15.3:0/4173245977 wait complete. 2024-08-01T04:18:22.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.251+0000 7fb432200640 1 Processor -- start 2024-08-01T04:18:22.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.251+0000 7fb432200640 1 -- start start 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb432200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c1016f0 0x7fb42c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb432200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 0x7fb42c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb432200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb42c1076f0 0x7fb42c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb432200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c111af0 con 0x7fb42c1016f0 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb432200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c111970 con 0x7fb42c1076f0 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb432200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c111c70 con 0x7fb42c102090 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb42be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 0x7fb42c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb430e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c1016f0 0x7fb42c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb42be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 0x7fb42c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:48326/0 (socket says 172.21.15.3:48326) 2024-08-01T04:18:22.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb42be00640 1 -- 172.21.15.3:0/4258347036 learned_addr learned my addr 172.21.15.3:0/4258347036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.252+0000 7fb431800640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb42c1076f0 0x7fb42c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb42be00640 1 -- 172.21.15.3:0/4258347036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb42c1076f0 msgr2=0x7fb42c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb42be00640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb42c1076f0 0x7fb42c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb42be00640 1 -- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c1016f0 msgr2=0x7fb42c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb42be00640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c1016f0 0x7fb42c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb42be00640 1 -- 172.21.15.3:0/4258347036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c19cf40 con 0x7fb42c102090 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb42be00640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 0x7fb42c1984d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb41800ea60 tx=0x7fb41800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb429600640 1 -- 172.21.15.3:0/4258347036 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800cde0 con 0x7fb42c102090 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb429600640 1 -- 172.21.15.3:0/4258347036 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4180035f0 con 0x7fb42c102090 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb429600640 1 -- 172.21.15.3:0/4258347036 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418010670 con 0x7fb42c102090 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c19d230 con 0x7fb42c102090 2024-08-01T04:18:22.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.253+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c1a4b10 con 0x7fb42c102090 2024-08-01T04:18:22.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.258+0000 7fb436444740 1 Processor -- start 2024-08-01T04:18:22.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.258+0000 7fb436444740 1 -- start start 2024-08-01T04:18:22.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb436444740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7427160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb436444740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b7431a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb436444740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b7434350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb436444740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c5b738cbf0 con 0x55c5b74276a0 2024-08-01T04:18:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb436444740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c5b7324bd0 con 0x55c5b7431f60 2024-08-01T04:18:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb436444740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c5b7388bb0 con 0x55c5b7426d80 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb429600640 1 -- 172.21.15.3:0/4258347036 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb418010810 con 0x7fb42c102090 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb431800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7427160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb431800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7427160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:48340/0 (socket says 172.21.15.3:48340) 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb431800640 1 -- 172.21.15.3:0/261626794 learned_addr learned my addr 172.21.15.3:0/261626794 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb430e00640 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b7431a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.259+0000 7fb42be00640 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b7434350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb431800640 1 -- 172.21.15.3:0/261626794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 msgr2=0x55c5b7434350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb431800640 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b7434350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb431800640 1 -- 172.21.15.3:0/261626794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 msgr2=0x55c5b7431a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb431800640 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b7431a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb431800640 1 -- 172.21.15.3:0/261626794 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5b732b8a0 con 0x55c5b7426d80 2024-08-01T04:18:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb430e00640 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b7431a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:22.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb431800640 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7427160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb424009cd0 tx=0x7fb424009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375c545f3fddfe31 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb41e000640 1 -- 172.21.15.3:0/261626794 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424048070 con 0x55c5b7426d80 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb41e000640 1 -- 172.21.15.3:0/261626794 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb424036070 con 0x55c5b7426d80 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.260+0000 7fb41e000640 1 -- 172.21.15.3:0/261626794 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424004400 con 0x55c5b7426d80 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb429600640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4140a2cf0 0x7fb4140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb436444740 1 -- 172.21.15.3:0/261626794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 msgr2=0x55c5b7427160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb436444740 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7427160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb424009cd0 tx=0x7fb424009a20 comp rx=0 tx=0).stop 2024-08-01T04:18:22.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb429600640 1 -- 172.21.15.3:0/4258347036 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb4180cf7b0 con 0x7fb42c102090 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb430e00640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4140a2cf0 0x7fb4140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb436444740 1 -- 172.21.15.3:0/261626794 shutdown_connections 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb436444740 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b7434350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb436444740 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b7431a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.261+0000 7fb436444740 1 --2- 172.21.15.3:0/261626794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7427160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.262+0000 7fb436444740 1 -- 172.21.15.3:0/261626794 >> 172.21.15.3:0/261626794 conn(0x55c5b7423310 msgr2=0x55c5b74236f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.262+0000 7fb436444740 1 -- 172.21.15.3:0/261626794 shutdown_connections 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.262+0000 7fb436444740 1 -- 172.21.15.3:0/261626794 wait complete. 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.262+0000 7fb436444740 1 -- start start 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.262+0000 7fb430e00640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4140a2cf0 0x7fb4140a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55c5b7424170 tx=0x7fb420005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:22.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.262+0000 7fb436444740 1 Processor -- start 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b742eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b742f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c5b7434b30 con 0x55c5b74276a0 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c5b7347d40 con 0x55c5b7431f60 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c5b7434cb0 con 0x55c5b7426d80 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb436444740 10 client.0 _ll_register_callbacks cb 0x55c5b7310b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb42be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b742f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb430e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b742eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb42be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b742f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37092/0 (socket says 172.21.15.3:37092) 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.263+0000 7fb42be00640 1 -- 172.21.15.3:0/3228905761 learned_addr learned my addr 172.21.15.3:0/3228905761 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:22.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb431800640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb42be00640 1 -- 172.21.15.3:0/3228905761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 msgr2=0x55c5b742eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb42be00640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b742eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb42be00640 1 -- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 msgr2=0x55c5b742f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb42be00640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb42be00640 1 -- 172.21.15.3:0/3228905761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5b70fe3c0 con 0x55c5b74276a0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb430e00640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b742eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb42be00640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b742f3e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb41800d010 tx=0x7fb418098020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a98b1c9d338892 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.264+0000 7fb412a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb412a00640 1 -- 172.21.15.3:0/3228905761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41809c390 con 0x55c5b74276a0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb412a00640 1 -- 172.21.15.3:0/3228905761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb4180d2e00 con 0x55c5b74276a0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb424036070 con 0x55c5b74276a0 2024-08-01T04:18:22.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb412a00640 1 -- 172.21.15.3:0/3228905761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4180b9050 con 0x55c5b74276a0 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb436444740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5b7440890 con 0x55c5b74276a0 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.265+0000 7fb412a00640 1 -- 172.21.15.3:0/3228905761 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb424036070 con 0x55c5b74276a0 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb412a00640 1 -- 172.21.15.3:0/3228905761 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb4180037a0 con 0x55c5b74276a0 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb436444740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb436444740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb436444740 20 client.15594 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb436444740 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b7441ca0 0x55c5b74440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb436444740 4 client.15594 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5b739a160 con 0x55c5b7441ca0 2024-08-01T04:18:22.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.266+0000 7fb430e00640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b7441ca0 0x55c5b74440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.267+0000 7fb430e00640 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b7441ca0 0x55c5b74440d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c031421ec3160df2 server_cookie=f50af9a572efa7d4 in_seq=0 out_seq=0 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.267+0000 7fb412a00640 10 client.15594 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb412a00640 1 -- 172.21.15.3:0/3228905761 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c5b739a160 con 0x55c5b7441ca0 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb412a00640 10 client.15594 handle_command_reply: tid=0 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 client.15594 shutdown 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 msgr2=0x55c5b742f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b742f3e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb41800d010 tx=0x7fb418098020 comp rx=0 tx=0).stop 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 shutdown_connections 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b7441ca0 0x55c5b74440d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b74276a0 0x55c5b742f3e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 --2- 172.21.15.3:0/3228905761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b742eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb412a00640 0 client.15594 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 >> 172.21.15.3:0/3228905761 conn(0x55c5b7423310 msgr2=0x55c5b7429040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:22.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 shutdown_connections 2024-08-01T04:18:22.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.268+0000 7fb436444740 1 -- 172.21.15.3:0/3228905761 wait complete. 2024-08-01T04:18:22.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.269+0000 7fb436444740 20 client.15594 trim_cache size 0 max 16384 2024-08-01T04:18:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 Processor -- start 2024-08-01T04:18:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 -- start start 2024-08-01T04:18:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7429830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b74276a0 0x55c5b7430e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 0x55c5b74499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1afde0 con 0x55c5b7431f60 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1ad6b0 con 0x55c5b74276a0 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.288+0000 7fb436444740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1ab500 con 0x55c5b7426d80 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 0x55c5b74499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 0x55c5b74499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37096/0 (socket says 172.21.15.3:37096) 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 -- 172.21.15.3:0/1263937372 learned_addr learned my addr 172.21.15.3:0/1263937372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb42be00640 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b74276a0 0x55c5b7430e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb430e00640 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7429830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 -- 172.21.15.3:0/1263937372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 msgr2=0x55c5b7429830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7429830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 -- 172.21.15.3:0/1263937372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b74276a0 msgr2=0x55c5b7430e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b74276a0 0x55c5b7430e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.289+0000 7fb431800640 1 -- 172.21.15.3:0/1263937372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5b74243c0 con 0x55c5b7431f60 2024-08-01T04:18:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.290+0000 7fb431800640 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 0x55c5b74499a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb424005fb0 tx=0x7fb424003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7c24bcb6b171af server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.290+0000 7fb41d600640 1 -- 172.21.15.3:0/1263937372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424048040 con 0x55c5b7431f60 2024-08-01T04:18:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.290+0000 7fb41d600640 1 -- 172.21.15.3:0/1263937372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb424036040 con 0x55c5b7431f60 2024-08-01T04:18:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.290+0000 7fb41d600640 1 -- 172.21.15.3:0/1263937372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424004590 con 0x55c5b7431f60 2024-08-01T04:18:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 -- 172.21.15.3:0/1263937372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 msgr2=0x55c5b74499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 0x55c5b74499a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb424005fb0 tx=0x7fb424003ef0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 -- 172.21.15.3:0/1263937372 shutdown_connections 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7431f60 0x55c5b74499a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b74276a0 0x55c5b7430e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 --2- 172.21.15.3:0/1263937372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b7426d80 0x55c5b7429830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 -- 172.21.15.3:0/1263937372 >> 172.21.15.3:0/1263937372 conn(0x55c5b7423310 msgr2=0x55c5b7430700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 -- 172.21.15.3:0/1263937372 shutdown_connections 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 -- 172.21.15.3:0/1263937372 wait complete. 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.291+0000 7fb436444740 1 -- start start 2024-08-01T04:18:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 Processor -- start 2024-08-01T04:18:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7426d80 0x55c5b743a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 0x55c5b742bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1abb70 con 0x55c5b7426d80 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1acfe0 con 0x55c5b7431f60 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1ac240 con 0x55c5b74276a0 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb436444740 10 client.0 _ll_register_callbacks cb 0x7fb418003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb430e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7426d80 0x55c5b743a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb42be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 0x55c5b742bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb42be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 0x55c5b742bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:48358/0 (socket says 172.21.15.3:48358) 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb430e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7426d80 0x55c5b743a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37100/0 (socket says 172.21.15.3:37100) 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.292+0000 7fb42be00640 1 -- 172.21.15.3:0/3491656074 learned_addr learned my addr 172.21.15.3:0/3491656074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb431800640 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb42be00640 1 -- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 msgr2=0x55c5b742c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb42be00640 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb42be00640 1 -- 172.21.15.3:0/3491656074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7426d80 msgr2=0x55c5b743a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb42be00640 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7426d80 0x55c5b743a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb42be00640 1 -- 172.21.15.3:0/3491656074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5b7423fa0 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb42be00640 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 0x55c5b742bc30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c760 tx=0x7fb418005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6006b66b3183d30 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb412a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb412a00640 1 -- 172.21.15.3:0/3491656074 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41809b7d0 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.293+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb424036040 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb412a00640 1 -- 172.21.15.3:0/3491656074 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb418086040 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb412a00640 1 -- 172.21.15.3:0/3491656074 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4180cfa80 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 10 client.25732 fetch_fsmap learned FSMap version 27 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5b74b0be0 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb412a00640 1 -- 172.21.15.3:0/3491656074 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb424036040 con 0x55c5b74276a0 2024-08-01T04:18:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb412a00640 1 -- 172.21.15.3:0/3491656074 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb418004000 con 0x55c5b74276a0 2024-08-01T04:18:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 10 client.25732 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:18:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 20 client.25732 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b74b12d0 0x55c5b74b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 4 client.25732 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:18:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.294+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5b7394670 con 0x55c5b74b12d0 2024-08-01T04:18:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.295+0000 7fb430e00640 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b74b12d0 0x55c5b74b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.295+0000 7fb430e00640 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b74b12d0 0x55c5b74b36f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b7f80203176ce0 server_cookie=e87161c4b5d190dd in_seq=0 out_seq=0 2024-08-01T04:18:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.295+0000 7fb412a00640 10 client.25732 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb412a00640 1 -- 172.21.15.3:0/3491656074 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c5b7394670 con 0x55c5b74b12d0 2024-08-01T04:18:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb412a00640 10 client.25732 handle_command_reply: tid=0 2024-08-01T04:18:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 client.25732 shutdown 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 msgr2=0x55c5b742bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 0x55c5b742bc30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c760 tx=0x7fb418005b20 comp rx=0 tx=0).stop 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 shutdown_connections 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c5b74b12d0 0x55c5b74b36f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c5b7431f60 0x55c5b742c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c5b74276a0 0x55c5b742bc30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 --2- 172.21.15.3:0/3491656074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c5b7426d80 0x55c5b743a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb412a00640 0 client.25732 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 >> 172.21.15.3:0/3491656074 conn(0x55c5b7423310 msgr2=0x55c5b742f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.297+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 shutdown_connections 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb436444740 1 -- 172.21.15.3:0/3491656074 wait complete. 2024-08-01T04:18:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb436444740 20 client.25732 trim_cache size 0 max 16384 2024-08-01T04:18:22.301 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: "0e13d647-c8c5-4399-9d5c-b0ff32a860fd": { 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "0e13d647-c8c5-4399-9d5c-b0ff32a860fd", 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:18:22.302 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:18:22.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4140a2cf0 msgr2=0x7fb4140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4140a2cf0 0x7fb4140a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55c5b7424170 tx=0x7fb420005f90 comp rx=0 tx=0).stop 2024-08-01T04:18:22.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 msgr2=0x7fb42c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:22.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 0x7fb42c1984d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb41800ea60 tx=0x7fb41800ef20 comp rx=0 tx=0).stop 2024-08-01T04:18:22.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 shutdown_connections 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb42c1076f0 0x7fb42c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.298+0000 7fb432200640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4140a2cf0 0x7fb4140a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.299+0000 7fb432200640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb42c102090 0x7fb42c1984d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.299+0000 7fb432200640 1 --2- 172.21.15.3:0/4258347036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb42c1016f0 0x7fb42c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.299+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 >> 172.21.15.3:0/4258347036 conn(0x7fb42c0fd410 msgr2=0x7fb42c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.299+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 shutdown_connections 2024-08-01T04:18:22.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:22.299+0000 7fb432200640 1 -- 172.21.15.3:0/4258347036 wait complete. 2024-08-01T04:18:22.849 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether build environment is sane... yes 2024-08-01T04:18:22.867 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-08-01T04:18:22.867 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gawk... gawk 2024-08-01T04:18:22.902 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether make sets $(MAKE)... yes 2024-08-01T04:18:23.012 INFO:tasks.workunit.client.0.smithi003.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-01T04:18:23.012 INFO:tasks.workunit.client.0.smithi003.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-01T04:18:23.035 INFO:tasks.workunit.client.0.smithi003.stdout:checking for style of include used by make... GNU 2024-08-01T04:18:23.041 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... gcc 2024-08-01T04:18:23.131 INFO:tasks.cephfs.filesystem:scrub status for tag:0e13d647-c8c5-4399-9d5c-b0ff32a860fd - {'path': '/', 'tag': '0e13d647-c8c5-4399-9d5c-b0ff32a860fd', 'options': 'recursive,force'} 2024-08-01T04:18:23.226 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether the C compiler works... yes 2024-08-01T04:18:23.226 INFO:tasks.workunit.client.0.smithi003.stdout:checking for C compiler default output file name... a.out 2024-08-01T04:18:23.309 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of executables... 2024-08-01T04:18:23.401 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are cross compiling... no 2024-08-01T04:18:23.475 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of object files... o 2024-08-01T04:18:23.534 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... yes 2024-08-01T04:18:23.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:23 smithi163 ceph-mon[37383]: pgmap v66: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 416 KiB/s wr, 48 op/s 2024-08-01T04:18:23.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:23 smithi003 ceph-mon[32262]: pgmap v66: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 416 KiB/s wr, 48 op/s 2024-08-01T04:18:23.594 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... yes 2024-08-01T04:18:23.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:23 smithi160 ceph-mon[37224]: pgmap v66: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 416 KiB/s wr, 48 op/s 2024-08-01T04:18:23.673 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... none needed 2024-08-01T04:18:23.859 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... gcc3 2024-08-01T04:18:23.964 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to run the C preprocessor... gcc -E 2024-08-01T04:18:24.072 INFO:tasks.workunit.client.0.smithi003.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-08-01T04:18:24.078 INFO:tasks.workunit.client.0.smithi003.stdout:checking for egrep... /usr/bin/grep -E 2024-08-01T04:18:24.346 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... yes 2024-08-01T04:18:24.424 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/types.h... yes 2024-08-01T04:18:24.516 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/stat.h... yes 2024-08-01T04:18:24.605 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdlib.h... yes 2024-08-01T04:18:24.703 INFO:tasks.workunit.client.0.smithi003.stdout:checking for string.h... yes 2024-08-01T04:18:24.800 INFO:tasks.workunit.client.0.smithi003.stdout:checking for memory.h... yes 2024-08-01T04:18:24.901 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strings.h... yes 2024-08-01T04:18:24.998 INFO:tasks.workunit.client.0.smithi003.stdout:checking for inttypes.h... yes 2024-08-01T04:18:25.096 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... yes 2024-08-01T04:18:25.193 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... yes 2024-08-01T04:18:25.288 INFO:tasks.workunit.client.0.smithi003.stdout:checking minix/config.h usability... no 2024-08-01T04:18:25.343 INFO:tasks.workunit.client.0.smithi003.stdout:checking minix/config.h presence... no 2024-08-01T04:18:25.343 INFO:tasks.workunit.client.0.smithi003.stdout:checking for minix/config.h... no 2024-08-01T04:18:25.423 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-08-01T04:18:25.424 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... (cached) gcc 2024-08-01T04:18:25.507 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-08-01T04:18:25.508 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... (cached) yes 2024-08-01T04:18:25.508 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-08-01T04:18:25.509 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... (cached) gcc3 2024-08-01T04:18:25.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:25 smithi163 ceph-mon[37383]: pgmap v67: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 303 KiB/s wr, 39 op/s 2024-08-01T04:18:25.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:25 smithi003 ceph-mon[32262]: pgmap v67: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 303 KiB/s wr, 39 op/s 2024-08-01T04:18:25.611 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lm... yes 2024-08-01T04:18:25.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:25 smithi160 ceph-mon[37224]: pgmap v67: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 303 KiB/s wr, 39 op/s 2024-08-01T04:18:25.721 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lpthread... yes 2024-08-01T04:18:25.723 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... (cached) yes 2024-08-01T04:18:25.808 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-08-01T04:18:25.926 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h usability... yes 2024-08-01T04:18:25.966 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h presence... yes 2024-08-01T04:18:25.966 INFO:tasks.workunit.client.0.smithi003.stdout:checking for pthread.h... yes 2024-08-01T04:18:26.057 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h usability... yes 2024-08-01T04:18:26.095 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h presence... yes 2024-08-01T04:18:26.095 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fcntl.h... yes 2024-08-01T04:18:26.201 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h usability... yes 2024-08-01T04:18:26.239 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h presence... yes 2024-08-01T04:18:26.239 INFO:tasks.workunit.client.0.smithi003.stdout:checking for limits.h... yes 2024-08-01T04:18:26.267 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... (cached) yes 2024-08-01T04:18:26.359 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h usability... yes 2024-08-01T04:18:26.398 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h presence... yes 2024-08-01T04:18:26.398 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/time.h... yes 2024-08-01T04:18:26.425 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... (cached) yes 2024-08-01T04:18:26.514 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h usability... yes 2024-08-01T04:18:26.555 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h presence... yes 2024-08-01T04:18:26.555 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/vfs.h... yes 2024-08-01T04:18:26.670 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h usability... no 2024-08-01T04:18:26.729 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h presence... no 2024-08-01T04:18:26.730 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/limits.h... no 2024-08-01T04:18:26.794 INFO:tasks.workunit.client.0.smithi003.stdout:checking for an ANSI C-conforming const... yes 2024-08-01T04:18:26.859 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-01T04:18:26.981 INFO:tasks.workunit.client.0.smithi003.stdout:checking for system... yes 2024-08-01T04:18:27.108 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gettimeofday... yes 2024-08-01T04:18:27.236 INFO:tasks.workunit.client.0.smithi003.stdout:checking for mkdir... yes 2024-08-01T04:18:27.356 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strerror... yes 2024-08-01T04:18:27.483 INFO:tasks.workunit.client.0.smithi003.stdout:checking for open64... yes 2024-08-01T04:18:27.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:27 smithi163 ceph-mon[37383]: pgmap v68: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 303 KiB/s wr, 39 op/s 2024-08-01T04:18:27.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:27 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:18:27.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:27 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:18:27.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:27 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:18:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:27 smithi003 ceph-mon[32262]: pgmap v68: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 303 KiB/s wr, 39 op/s 2024-08-01T04:18:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:27 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:18:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:27 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:18:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:27 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:18:27.612 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stat64... yes 2024-08-01T04:18:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:27 smithi160 ceph-mon[37224]: pgmap v68: 97 pgs: 97 active+clean; 5.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 303 KiB/s wr, 39 op/s 2024-08-01T04:18:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:27 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:18:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:27 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:18:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:27 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:18:27.744 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fseeko64... yes 2024-08-01T04:18:27.873 INFO:tasks.workunit.client.0.smithi003.stdout:checking for lrand48_r... yes 2024-08-01T04:18:27.997 INFO:tasks.workunit.client.0.smithi003.stdout:checking for srand48_r... yes 2024-08-01T04:18:28.047 INFO:tasks.workunit.client.0.smithi003.stdout:configure: creating ./config.status 2024-08-01T04:18:28.233 INFO:tasks.workunit.client.0.smithi003.stdout: /bin/sh ./config.status 2024-08-01T04:18:28.367 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating Makefile 2024-08-01T04:18:28.407 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating config.h 2024-08-01T04:18:28.456 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing depfiles commands 2024-08-01T04:18:28.501 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing default commands 2024-08-01T04:18:28.538 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-08-01T04:18:28.798 INFO:tasks.workunit.client.0.smithi003.stdout:rm -f stamp-h1 2024-08-01T04:18:28.802 INFO:tasks.workunit.client.0.smithi003.stdout:touch config.h.in 2024-08-01T04:18:28.806 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh ./config.status config.h 2024-08-01T04:18:28.915 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating config.h 2024-08-01T04:18:28.969 INFO:tasks.workunit.client.0.smithi003.stdout:make all-am 2024-08-01T04:18:28.979 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-08-01T04:18:28.979 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-08-01T04:18:29.254 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-08-01T04:18:29.261 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-08-01T04:18:29.392 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-08-01T04:18:29.398 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-08-01T04:18:29.533 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-08-01T04:18:29.540 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-08-01T04:18:29.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:29 smithi163 ceph-mon[37383]: pgmap v69: 97 pgs: 97 active+clean; 7.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 419 KiB/s wr, 48 op/s 2024-08-01T04:18:29.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:29 smithi003 ceph-mon[32262]: pgmap v69: 97 pgs: 97 active+clean; 7.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 419 KiB/s wr, 48 op/s 2024-08-01T04:18:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:29 smithi160 ceph-mon[37224]: pgmap v69: 97 pgs: 97 active+clean; 7.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 419 KiB/s wr, 48 op/s 2024-08-01T04:18:29.681 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-08-01T04:18:29.688 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-08-01T04:18:29.876 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-08-01T04:18:29.882 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-08-01T04:18:29.977 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-08-01T04:18:29.983 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-08-01T04:18:30.064 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-08-01T04:18:30.071 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-08-01T04:18:30.159 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-08-01T04:18:30.166 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-08-01T04:18:30.442 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-08-01T04:18:30.449 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-08-01T04:18:30.644 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-08-01T04:18:30.650 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-08-01T04:18:30.864 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-08-01T04:18:30.871 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-08-01T04:18:30.987 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-08-01T04:18:30.994 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-08-01T04:18:31.115 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-08-01T04:18:31.123 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-08-01T04:18:31.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:31 smithi163 ceph-mon[37383]: pgmap v70: 97 pgs: 97 active+clean; 7.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 212 KiB/s wr, 18 op/s 2024-08-01T04:18:31.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:31 smithi003 ceph-mon[32262]: pgmap v70: 97 pgs: 97 active+clean; 7.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 212 KiB/s wr, 18 op/s 2024-08-01T04:18:31.582 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-08-01T04:18:31.589 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr:In file included from parser.h:22, 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr: from parser.c:25: 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘init_filesys’: 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.648 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-08-01T04:18:31.649 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.651 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.652 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.653 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘init_tg_stats’: 2024-08-01T04:18:31.654 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.655 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: 971 | list_for_each_entry(tmp_list, 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.656 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: 971 | list_for_each_entry(tmp_list, 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.657 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: 980 | list_for_each_entry(tmp_list, 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-08-01T04:18:31.658 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: 980 | list_for_each_entry(tmp_list, 2024-08-01T04:18:31.662 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:31 smithi160 ceph-mon[37224]: pgmap v70: 97 pgs: 97 active+clean; 7.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 212 KiB/s wr, 18 op/s 2024-08-01T04:18:31.765 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘get_optstr’: 2024-08-01T04:18:31.765 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-08-01T04:18:31.765 INFO:tasks.workunit.client.0.smithi003.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-08-01T04:18:31.765 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:31.766 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:200:14: note: source object declared here 2024-08-01T04:18:31.766 INFO:tasks.workunit.client.0.smithi003.stderr: 200 | char temp[BUFSIZE]; 2024-08-01T04:18:31.766 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~ 2024-08-01T04:18:31.886 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘size64_convert’: 2024-08-01T04:18:31.886 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-08-01T04:18:31.886 INFO:tasks.workunit.client.0.smithi003.stderr: 119 | memcpy(unit, "\0", 3); 2024-08-01T04:18:31.886 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-08-01T04:18:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-08-01T04:18:32.271 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-08-01T04:18:32.379 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-08-01T04:18:32.385 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-08-01T04:18:32.538 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-08-01T04:18:32.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:32 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:18:32.546 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-08-01T04:18:32.547 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:32 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:18:32.601 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-08-01T04:18:32.607 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-08-01T04:18:32.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:32 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:18:32.662 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-08-01T04:18:32.663 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd .. 2024-08-01T04:18:32.663 INFO:tasks.workunit.client.0.smithi003.stderr:+ mkdir tmp 2024-08-01T04:18:32.668 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd tmp 2024-08-01T04:18:32.669 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in $mydir/*.ffsb 2024-08-01T04:18:32.669 INFO:tasks.workunit.client.0.smithi003.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-08-01T04:18:32.671 INFO:tasks.workunit.client.0.smithi003.stdout:FFSB version 6.0-RC2 started 2024-08-01T04:18:32.671 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.677 INFO:tasks.workunit.client.0.smithi003.stdout:benchmark time = 300 2024-08-01T04:18:32.677 INFO:tasks.workunit.client.0.smithi003.stdout:ThreadGroup 0 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout:================ 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: num_threads = 32 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: read_random = off 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: read_size = 0 (0B) 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: read_blocksize = 0 (0B) 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: read_skip = off 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: read_skipsize = 0 (0B) 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: write_random = on 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: write_size = 5242880 (5MB) 2024-08-01T04:18:32.678 INFO:tasks.workunit.client.0.smithi003.stdout: fsync_file = 0 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: write_blocksize = 4096 (4KB) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: wait time = 0 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: op weights 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: read = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: readall = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: write = 1 (100.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: create = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: append = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: delete = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: metaop = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: createdir = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: stat = 0 (0.00%) 2024-08-01T04:18:32.679 INFO:tasks.workunit.client.0.smithi003.stdout: writeall = 0 (0.00%) 2024-08-01T04:18:32.680 INFO:tasks.workunit.client.0.smithi003.stdout: writeall_fsync = 0 (0.00%) 2024-08-01T04:18:32.680 INFO:tasks.workunit.client.0.smithi003.stdout: open_close = 0 (0.00%) 2024-08-01T04:18:32.680 INFO:tasks.workunit.client.0.smithi003.stdout: write_fsync = 0 (0.00%) 2024-08-01T04:18:32.680 INFO:tasks.workunit.client.0.smithi003.stdout: create_fsync = 0 (0.00%) 2024-08-01T04:18:32.681 INFO:tasks.workunit.client.0.smithi003.stdout: append_fsync = 0 (0.00%) 2024-08-01T04:18:32.681 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.681 INFO:tasks.workunit.client.0.smithi003.stdout:FileSystem . 2024-08-01T04:18:32.681 INFO:tasks.workunit.client.0.smithi003.stdout:========== 2024-08-01T04:18:32.681 INFO:tasks.workunit.client.0.smithi003.stdout: num_dirs = 0 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: starting files = 128 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: min file size = 104857600 (100MB) 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: max file size = 104857600 (100MB) 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: directio = off 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: alignedio = on 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: bufferedio = off 2024-08-01T04:18:32.682 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.683 INFO:tasks.workunit.client.0.smithi003.stdout: aging is off 2024-08-01T04:18:32.683 INFO:tasks.workunit.client.0.smithi003.stdout: current utilization = 0.00% 2024-08-01T04:18:32.683 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:18:32.683 INFO:tasks.workunit.client.0.smithi003.stdout:checking existing fs: . 2024-08-01T04:18:32.683 INFO:tasks.workunit.client.0.smithi003.stdout:recreating new fileset 2024-08-01T04:18:32.683 INFO:tasks.workunit.client.0.smithi003.stderr:opendir: No such file or directory 2024-08-01T04:18:33.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:33 smithi163 ceph-mon[37383]: pgmap v71: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 270 KiB/s wr, 25 op/s 2024-08-01T04:18:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:33 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:18:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:33 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:18:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:33 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:18:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:33 smithi003 ceph-mon[32262]: pgmap v71: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 270 KiB/s wr, 25 op/s 2024-08-01T04:18:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:33 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:18:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:33 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:18:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:33 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:18:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:33 smithi160 ceph-mon[37224]: pgmap v71: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 270 KiB/s wr, 25 op/s 2024-08-01T04:18:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:33 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:18:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:33 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:18:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:33 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:18:35.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:35 smithi163 ceph-mon[37383]: pgmap v72: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 174 KiB/s wr, 15 op/s 2024-08-01T04:18:35.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:35 smithi003 ceph-mon[32262]: pgmap v72: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 174 KiB/s wr, 15 op/s 2024-08-01T04:18:35.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:35 smithi160 ceph-mon[37224]: pgmap v72: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 174 KiB/s wr, 15 op/s 2024-08-01T04:18:37.488 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:37 smithi003 ceph-mon[32262]: pgmap v73: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 174 KiB/s wr, 15 op/s 2024-08-01T04:18:37.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:37 smithi163 ceph-mon[37383]: pgmap v73: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 174 KiB/s wr, 15 op/s 2024-08-01T04:18:37.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:37 smithi160 ceph-mon[37224]: pgmap v73: 97 pgs: 97 active+clean; 7.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 174 KiB/s wr, 15 op/s 2024-08-01T04:18:39.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:39 smithi163 ceph-mon[37383]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.8 MiB/s wr, 54 op/s 2024-08-01T04:18:39.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:39 smithi003 ceph-mon[32262]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.8 MiB/s wr, 54 op/s 2024-08-01T04:18:39.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:39 smithi160 ceph-mon[37224]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.8 MiB/s wr, 54 op/s 2024-08-01T04:18:41.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:41 smithi163 ceph-mon[37383]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.7 MiB/s wr, 46 op/s 2024-08-01T04:18:41.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:41 smithi003 ceph-mon[32262]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.7 MiB/s wr, 46 op/s 2024-08-01T04:18:41.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:41 smithi160 ceph-mon[37224]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.7 MiB/s wr, 46 op/s 2024-08-01T04:18:43.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:43 smithi163 ceph-mon[37383]: pgmap v76: 97 pgs: 97 active+clean; 107 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 8.4 MiB/s wr, 76 op/s 2024-08-01T04:18:43.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:43 smithi003 ceph-mon[32262]: pgmap v76: 97 pgs: 97 active+clean; 107 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 8.4 MiB/s wr, 76 op/s 2024-08-01T04:18:43.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:43 smithi160 ceph-mon[37224]: pgmap v76: 97 pgs: 97 active+clean; 107 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 8.4 MiB/s wr, 76 op/s 2024-08-01T04:18:45.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:45 smithi163 ceph-mon[37383]: pgmap v77: 97 pgs: 97 active+clean; 115 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 9.0 MiB/s wr, 72 op/s 2024-08-01T04:18:45.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:45 smithi003 ceph-mon[32262]: pgmap v77: 97 pgs: 97 active+clean; 115 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 9.0 MiB/s wr, 72 op/s 2024-08-01T04:18:45.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:45 smithi160 ceph-mon[37224]: pgmap v77: 97 pgs: 97 active+clean; 115 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 9.0 MiB/s wr, 72 op/s 2024-08-01T04:18:47.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:47 smithi163 ceph-mon[37383]: pgmap v78: 97 pgs: 97 active+clean; 115 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 9.0 MiB/s wr, 72 op/s 2024-08-01T04:18:47.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:47 smithi003 ceph-mon[32262]: pgmap v78: 97 pgs: 97 active+clean; 115 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 9.0 MiB/s wr, 72 op/s 2024-08-01T04:18:47.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:47 smithi160 ceph-mon[37224]: pgmap v78: 97 pgs: 97 active+clean; 115 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 9.0 MiB/s wr, 72 op/s 2024-08-01T04:18:49.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:49 smithi163 ceph-mon[37383]: pgmap v79: 97 pgs: 97 active+clean; 1.2 GiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 102 MiB/s wr, 110 op/s 2024-08-01T04:18:49.551 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:49 smithi003 ceph-mon[32262]: pgmap v79: 97 pgs: 97 active+clean; 1.2 GiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 102 MiB/s wr, 110 op/s 2024-08-01T04:18:49.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:49 smithi160 ceph-mon[37224]: pgmap v79: 97 pgs: 97 active+clean; 1.2 GiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 102 MiB/s wr, 110 op/s 2024-08-01T04:18:51.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:51 smithi160 ceph-mon[37224]: pgmap v80: 97 pgs: 97 active+clean; 1.2 GiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 100 MiB/s wr, 71 op/s 2024-08-01T04:18:51.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:51 smithi163 ceph-mon[37383]: pgmap v80: 97 pgs: 97 active+clean; 1.2 GiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 100 MiB/s wr, 71 op/s 2024-08-01T04:18:51.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:51 smithi003 ceph-mon[32262]: pgmap v80: 97 pgs: 97 active+clean; 1.2 GiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 100 MiB/s wr, 71 op/s 2024-08-01T04:18:53.132 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:18:53.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:53 smithi003 ceph-mon[32262]: pgmap v81: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 199 MiB/s wr, 109 op/s 2024-08-01T04:18:53.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:53 smithi160 ceph-mon[37224]: pgmap v81: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 199 MiB/s wr, 109 op/s 2024-08-01T04:18:53.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:53 smithi163 ceph-mon[37383]: pgmap v81: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 199 MiB/s wr, 109 op/s 2024-08-01T04:18:54.010 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.961+0000 7fbc5a400640 1 -- 172.21.15.3:0/2569728894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc54103900 msgr2=0x7fbc54103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.961+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2569728894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc54103900 0x7fbc54103d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009930 tx=0x7fbc4002a440 comp rx=0 tx=0).stop 2024-08-01T04:18:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.962+0000 7fbc5a400640 1 -- 172.21.15.3:0/2569728894 shutdown_connections 2024-08-01T04:18:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.962+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2569728894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc541042a0 0x7fbc541107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.962+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2569728894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc54103900 0x7fbc54103d60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.962+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2569728894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc54109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.962+0000 7fbc5a400640 1 -- 172.21.15.3:0/2569728894 >> 172.21.15.3:0/2569728894 conn(0x7fbc540ff640 msgr2=0x7fbc54101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.966+0000 7fbc5a400640 1 -- 172.21.15.3:0/2569728894 shutdown_connections 2024-08-01T04:18:54.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.966+0000 7fbc5a400640 1 -- 172.21.15.3:0/2569728894 wait complete. 2024-08-01T04:18:54.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.968+0000 7fbc5a400640 1 Processor -- start 2024-08-01T04:18:54.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.968+0000 7fbc5a400640 1 -- start start 2024-08-01T04:18:54.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.969+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc54103900 0x7fbc5419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.969+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc541042a0 0x7fbc5419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.969+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc5419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.969+0000 7fbc5a400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54113d50 con 0x7fbc54109900 2024-08-01T04:18:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.969+0000 7fbc5a400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54113bd0 con 0x7fbc54103900 2024-08-01T04:18:54.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.969+0000 7fbc5a400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54113ed0 con 0x7fbc541042a0 2024-08-01T04:18:54.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.970+0000 7fbc59a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc5419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:54.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.970+0000 7fbc59a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc5419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36986/0 (socket says 172.21.15.3:36986) 2024-08-01T04:18:54.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.970+0000 7fbc59a00640 1 -- 172.21.15.3:0/2848168651 learned_addr learned my addr 172.21.15.3:0/2848168651 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:54.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.971+0000 7fbc59a00640 1 -- 172.21.15.3:0/2848168651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc541042a0 msgr2=0x7fbc5419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:54.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.971+0000 7fbc59a00640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc541042a0 0x7fbc5419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.971+0000 7fbc59a00640 1 -- 172.21.15.3:0/2848168651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc54103900 msgr2=0x7fbc5419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:54.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.971+0000 7fbc59a00640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc54103900 0x7fbc5419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.971+0000 7fbc59a00640 1 -- 172.21.15.3:0/2848168651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5419f1c0 con 0x7fbc54109900 2024-08-01T04:18:54.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.972+0000 7fbc59a00640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc5419eae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00d750 tx=0x7fbc4c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:54.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.974+0000 7fbc51600640 1 -- 172.21.15.3:0/2848168651 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c017070 con 0x7fbc54109900 2024-08-01T04:18:54.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.974+0000 7fbc51600640 1 -- 172.21.15.3:0/2848168651 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc4c005500 con 0x7fbc54109900 2024-08-01T04:18:54.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.974+0000 7fbc51600640 1 -- 172.21.15.3:0/2848168651 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c004d90 con 0x7fbc54109900 2024-08-01T04:18:54.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.974+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5419f4b0 con 0x7fbc54109900 2024-08-01T04:18:54.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.974+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541a6d90 con 0x7fbc54109900 2024-08-01T04:18:54.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.983+0000 7fbc51600640 1 -- 172.21.15.3:0/2848168651 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fbc4c004f30 con 0x7fbc54109900 2024-08-01T04:18:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.987+0000 7fbc51600640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbc380a2e40 0x7fbc380a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.987+0000 7fbc51600640 1 -- 172.21.15.3:0/2848168651 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbc4c0cee00 con 0x7fbc54109900 2024-08-01T04:18:54.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.987+0000 7fbc59000640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbc380a2e40 0x7fbc380a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:54.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.990+0000 7fbc5e425740 1 Processor -- start 2024-08-01T04:18:54.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.991+0000 7fbc5e425740 1 -- start start 2024-08-01T04:18:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.991+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b436a0 0x560610b4da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b4df60 0x560610b50350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc5e425740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560610aa8bf0 con 0x560610b436a0 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc5e425740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560610a40bd0 con 0x560610b42d80 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc5e425740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560610aa4bb0 con 0x560610b4df60 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.991+0000 7fbc59000640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbc380a2e40 0x7fbc380a5300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005fd0 tx=0x7fbc44009940 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc59000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.992+0000 7fbc59a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b4df60 0x560610b50350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b4df60 0x560610b50350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41142/0 (socket says 172.21.15.3:41142) 2024-08-01T04:18:54.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59a00640 1 -- 172.21.15.3:0/1928469157 learned_addr learned my addr 172.21.15.3:0/1928469157 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc53e00640 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b436a0 0x560610b4da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59000640 1 -- 172.21.15.3:0/1928469157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b4df60 msgr2=0x560610b50350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:54.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59000640 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b4df60 0x560610b50350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59000640 1 -- 172.21.15.3:0/1928469157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b436a0 msgr2=0x560610b4da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59000640 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b436a0 0x560610b4da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.993+0000 7fbc59000640 1 -- 172.21.15.3:0/1928469157 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560610a478a0 con 0x560610b42d80 2024-08-01T04:18:54.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.994+0000 7fbc59000640 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b43160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400efd0 tx=0x7fbc44011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d89789f029d22a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.995+0000 7fbc4a000640 1 -- 172.21.15.3:0/1928469157 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44021070 con 0x560610b42d80 2024-08-01T04:18:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.995+0000 7fbc4a000640 1 -- 172.21.15.3:0/1928469157 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44010e40 con 0x560610b42d80 2024-08-01T04:18:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.995+0000 7fbc4a000640 1 -- 172.21.15.3:0/1928469157 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4401c990 con 0x560610b42d80 2024-08-01T04:18:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.996+0000 7fbc5e425740 1 -- 172.21.15.3:0/1928469157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 msgr2=0x560610b43160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:54.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.996+0000 7fbc5e425740 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b43160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400efd0 tx=0x7fbc44011000 comp rx=0 tx=0).stop 2024-08-01T04:18:54.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.997+0000 7fbc5e425740 1 -- 172.21.15.3:0/1928469157 shutdown_connections 2024-08-01T04:18:54.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.997+0000 7fbc5e425740 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b4df60 0x560610b50350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:54.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.997+0000 7fbc5e425740 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b436a0 0x560610b4da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.997+0000 7fbc5e425740 1 --2- 172.21.15.3:0/1928469157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b43160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.997+0000 7fbc5e425740 1 -- 172.21.15.3:0/1928469157 >> 172.21.15.3:0/1928469157 conn(0x560610b3f310 msgr2=0x560610b3f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.998+0000 7fbc5e425740 1 -- 172.21.15.3:0/1928469157 shutdown_connections 2024-08-01T04:18:55.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.998+0000 7fbc5e425740 1 -- 172.21.15.3:0/1928469157 wait complete. 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.999+0000 7fbc5e425740 1 -- start start 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:54.999+0000 7fbc5e425740 1 Processor -- start 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b4aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b4b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc5e425740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560610b61b40 con 0x560610b4df60 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc5e425740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560610a63d40 con 0x560610b42d80 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc5e425740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x560610b61cc0 con 0x560610b436a0 2024-08-01T04:18:55.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc59000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b4aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc59000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b4aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:35990/0 (socket says 172.21.15.3:35990) 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc59000640 1 -- 172.21.15.3:0/3068607817 learned_addr learned my addr 172.21.15.3:0/3068607817 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.000+0000 7fbc59a00640 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b4b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.001+0000 7fbc5e425740 10 client.0 _ll_register_callbacks cb 0x560610a2cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.001+0000 7fbc59a00640 1 -- 172.21.15.3:0/3068607817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 msgr2=0x560610b4b400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.001+0000 7fbc59a00640 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.001+0000 7fbc59a00640 1 -- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 msgr2=0x560610b4aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.001+0000 7fbc59a00640 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b4aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.001+0000 7fbc59a00640 1 -- 172.21.15.3:0/3068607817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56061081a3c0 con 0x560610b4df60 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.002+0000 7fbc59a00640 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b4b940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c09b270 tx=0x7fbc4c09ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c6492e96c6bca3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.002+0000 7fbc3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.002+0000 7fbc3ea00640 1 -- 172.21.15.3:0/3068607817 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0b4060 con 0x560610b4df60 2024-08-01T04:18:55.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.003+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560610ab6160 con 0x560610b4df60 2024-08-01T04:18:55.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.003+0000 7fbc3ea00640 1 -- 172.21.15.3:0/3068607817 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbc4c0cc790 con 0x560610b4df60 2024-08-01T04:18:55.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.003+0000 7fbc3ea00640 1 -- 172.21.15.3:0/3068607817 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c0b9050 con 0x560610b4df60 2024-08-01T04:18:55.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.003+0000 7fbc5e425740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-08-01T04:18:55.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.003+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560610b509a0 con 0x560610b4df60 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc3ea00640 1 -- 172.21.15.3:0/3068607817 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560610ab6160 con 0x560610b4df60 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc3ea00640 1 -- 172.21.15.3:0/3068607817 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbc4c09f170 con 0x560610b4df60 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc5e425740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc5e425740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc5e425740 20 client.15612 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc5e425740 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610b51db0 0x560610b54240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc5e425740 4 client.15612 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560610ab0670 con 0x560610b51db0 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.004+0000 7fbc59000640 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610b51db0 0x560610b54240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.005+0000 7fbc59000640 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610b51db0 0x560610b54240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729d5aaba6e3b5e2 server_cookie=cd8318e57da8d85 in_seq=0 out_seq=0 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.005+0000 7fbc3ea00640 10 client.15612 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:55.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc3ea00640 1 -- 172.21.15.3:0/3068607817 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560610ab0670 con 0x560610b51db0 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc3ea00640 10 client.15612 handle_command_reply: tid=0 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 client.15612 shutdown 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 msgr2=0x560610b4b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b4b940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c09b270 tx=0x7fbc4c09ad30 comp rx=0 tx=0).stop 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 shutdown_connections 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610b51db0 0x560610b54240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b4b940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 --2- 172.21.15.3:0/3068607817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b4aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc3ea00640 0 client.15612 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.006+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 >> 172.21.15.3:0/3068607817 conn(0x560610b3f310 msgr2=0x560610b45040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.007+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 shutdown_connections 2024-08-01T04:18:55.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.007+0000 7fbc5e425740 1 -- 172.21.15.3:0/3068607817 wait complete. 2024-08-01T04:18:55.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.007+0000 7fbc5e425740 20 client.15612 trim_cache size 0 max 16384 2024-08-01T04:18:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 Processor -- start 2024-08-01T04:18:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 -- start start 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b3e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541a99b0 con 0x560610b4df60 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbc541a7280 con 0x560610b42d80 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.026+0000 7fbc5e425740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbc540722c0 con 0x560610b436a0 2024-08-01T04:18:55.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.027+0000 7fbc53e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.027+0000 7fbc53e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41144/0 (socket says 172.21.15.3:41144) 2024-08-01T04:18:55.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.027+0000 7fbc53e00640 1 -- 172.21.15.3:0/2024699165 learned_addr learned my addr 172.21.15.3:0/2024699165 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:55.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.027+0000 7fbc59a00640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b3e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.027+0000 7fbc53e00640 1 -- 172.21.15.3:0/2024699165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 msgr2=0x560610b457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:55.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.027+0000 7fbc59000640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc53e00640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc53e00640 1 -- 172.21.15.3:0/2024699165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 msgr2=0x560610b3e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc53e00640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b3e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc53e00640 1 -- 172.21.15.3:0/2024699165 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560610b403c0 con 0x560610b436a0 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc59000640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc59a00640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b3e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc53e00640 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc54101c00 tx=0x7fbc40005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1681dc2cf7698 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.028+0000 7fbc49600640 1 -- 172.21.15.3:0/2024699165 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40040070 con 0x560610b436a0 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.029+0000 7fbc49600640 1 -- 172.21.15.3:0/2024699165 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40004230 con 0x560610b436a0 2024-08-01T04:18:55.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.029+0000 7fbc49600640 1 -- 172.21.15.3:0/2024699165 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40035040 con 0x560610b436a0 2024-08-01T04:18:55.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.029+0000 7fbc5e425740 1 -- 172.21.15.3:0/2024699165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 msgr2=0x560610b4b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:55.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.029+0000 7fbc5e425740 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc54101c00 tx=0x7fbc40005d50 comp rx=0 tx=0).stop 2024-08-01T04:18:55.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 -- 172.21.15.3:0/2024699165 shutdown_connections 2024-08-01T04:18:55.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b4df60 0x560610b3e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b4b790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 --2- 172.21.15.3:0/2024699165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b42d80 0x560610b457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 -- 172.21.15.3:0/2024699165 >> 172.21.15.3:0/2024699165 conn(0x560610b3f310 msgr2=0x560610b3dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:55.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 -- 172.21.15.3:0/2024699165 shutdown_connections 2024-08-01T04:18:55.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.030+0000 7fbc5e425740 1 -- 172.21.15.3:0/2024699165 wait complete. 2024-08-01T04:18:55.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.031+0000 7fbc5e425740 1 -- start start 2024-08-01T04:18:55.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.031+0000 7fbc5e425740 1 Processor -- start 2024-08-01T04:18:55.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b42d80 0x560610b48880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b66f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b4df60 0x560610b67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54072930 con 0x560610b42d80 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54073da0 con 0x560610b4df60 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbc54073000 con 0x560610b436a0 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.032+0000 7fbc5e425740 10 client.0 _ll_register_callbacks cb 0x7fbc4c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b42d80 0x560610b48880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b42d80 0x560610b48880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37024/0 (socket says 172.21.15.3:37024) 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59000640 1 -- 172.21.15.3:0/776930646 learned_addr learned my addr 172.21.15.3:0/776930646 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:55.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59a00640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b4df60 0x560610b67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc53e00640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b66f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59a00640 1 -- 172.21.15.3:0/776930646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 msgr2=0x560610b66f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:55.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59a00640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b66f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59a00640 1 -- 172.21.15.3:0/776930646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b42d80 msgr2=0x560610b48880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59a00640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b42d80 0x560610b48880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.033+0000 7fbc59a00640 1 -- 172.21.15.3:0/776930646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560610b67a40 con 0x560610b4df60 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.034+0000 7fbc59a00640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b4df60 0x560610b67450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0cc5c0 tx=0x7fbc4c097bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92aa67db1869784a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.034+0000 7fbc3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.034+0000 7fbc3ea00640 1 -- 172.21.15.3:0/776930646 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c020040 con 0x560610b4df60 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.034+0000 7fbc3ea00640 1 -- 172.21.15.3:0/776930646 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbc4c0b4060 con 0x560610b4df60 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.034+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc40004230 con 0x560610b4df60 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.035+0000 7fbc3ea00640 1 -- 172.21.15.3:0/776930646 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc4c09ae20 con 0x560610b4df60 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.035+0000 7fbc5e425740 10 client.25757 fetch_fsmap learned FSMap version 27 2024-08-01T04:18:55.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.035+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560610bc89f0 con 0x560610b4df60 2024-08-01T04:18:55.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.035+0000 7fbc3ea00640 1 -- 172.21.15.3:0/776930646 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40004230 con 0x560610b4df60 2024-08-01T04:18:55.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.036+0000 7fbc3ea00640 1 -- 172.21.15.3:0/776930646 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbc4c09a730 con 0x560610b4df60 2024-08-01T04:18:55.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.036+0000 7fbc5e425740 10 client.25757 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:18:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.036+0000 7fbc5e425740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:18:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.036+0000 7fbc5e425740 20 client.25757 populate_metadata read hostname 'smithi003' 2024-08-01T04:18:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.036+0000 7fbc5e425740 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610bc8eb0 0x560610bcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.037+0000 7fbc5e425740 4 client.25757 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:18:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.037+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560610a2d0c0 con 0x560610bc8eb0 2024-08-01T04:18:55.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.037+0000 7fbc59000640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610bc8eb0 0x560610bcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:55.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.038+0000 7fbc59000640 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610bc8eb0 0x560610bcb2d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53930d0a687607 server_cookie=fb7ba9c9f9a5467c in_seq=0 out_seq=0 2024-08-01T04:18:55.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.038+0000 7fbc3ea00640 10 client.25757 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:55.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.038+0000 7fbc3ea00640 1 -- 172.21.15.3:0/776930646 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560610a2d0c0 con 0x560610bc8eb0 2024-08-01T04:18:55.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.038+0000 7fbc3ea00640 10 client.25757 handle_command_reply: tid=0 2024-08-01T04:18:55.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.038+0000 7fbc5e425740 1 client.25757 shutdown 2024-08-01T04:18:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.039+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b4df60 msgr2=0x560610b67450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:55.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.039+0000 7fbc5e425740 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b4df60 0x560610b67450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0cc5c0 tx=0x7fbc4c097bf0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 shutdown_connections 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x560610bc8eb0 0x560610bcb2d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560610b4df60 0x560610b67450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560610b436a0 0x560610b66f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 --2- 172.21.15.3:0/776930646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560610b42d80 0x560610b48880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc3ea00640 0 client.25757 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 >> 172.21.15.3:0/776930646 conn(0x560610b3f310 msgr2=0x560610b3e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:55.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 shutdown_connections 2024-08-01T04:18:55.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 1 -- 172.21.15.3:0/776930646 wait complete. 2024-08-01T04:18:55.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.041+0000 7fbc5e425740 20 client.25757 trim_cache size 0 max 16384 2024-08-01T04:18:55.044 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:18:55.044 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:18:55.044 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:18:55.044 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.042+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbc380a2e40 msgr2=0x7fbc380a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.042+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbc380a2e40 0x7fbc380a5300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005fd0 tx=0x7fbc44009940 comp rx=0 tx=0).stop 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.042+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 msgr2=0x7fbc5419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.042+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc5419eae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00d750 tx=0x7fbc4c00dc10 comp rx=0 tx=0).stop 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 shutdown_connections 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc54109900 0x7fbc5419eae0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbc380a2e40 0x7fbc380a5300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc541042a0 0x7fbc5419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 --2- 172.21.15.3:0/2848168651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc54103900 0x7fbc5419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:55.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 >> 172.21.15.3:0/2848168651 conn(0x7fbc540ff640 msgr2=0x7fbc540ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:55.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 shutdown_connections 2024-08-01T04:18:55.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:55.043+0000 7fbc5a400640 1 -- 172.21.15.3:0/2848168651 wait complete. 2024-08-01T04:18:55.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:55 smithi003 ceph-mon[32262]: pgmap v82: 97 pgs: 97 active+clean; 2.5 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 206 MiB/s wr, 82 op/s 2024-08-01T04:18:55.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:55 smithi160 ceph-mon[37224]: pgmap v82: 97 pgs: 97 active+clean; 2.5 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 206 MiB/s wr, 82 op/s 2024-08-01T04:18:55.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:55 smithi163 ceph-mon[37383]: pgmap v82: 97 pgs: 97 active+clean; 2.5 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 206 MiB/s wr, 82 op/s 2024-08-01T04:18:56.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:18:56.129 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:18:56.830 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:18:57.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:57 smithi003 ceph-mon[32262]: pgmap v83: 97 pgs: 97 active+clean; 2.5 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 205 MiB/s wr, 78 op/s 2024-08-01T04:18:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.611+0000 7f06f7200640 1 -- 172.21.15.3:0/3698475364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 msgr2=0x7f06f810e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:57.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.611+0000 7f06f7200640 1 --2- 172.21.15.3:0/3698475364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f810e5c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009cd0 tx=0x7f06f002a910 comp rx=0 tx=0).stop 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.611+0000 7f06f7200640 1 -- 172.21.15.3:0/3698475364 shutdown_connections 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.611+0000 7f06f7200640 1 --2- 172.21.15.3:0/3698475364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f810e5c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.611+0000 7f06f7200640 1 --2- 172.21.15.3:0/3698475364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81016f0 0x7f06f8101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.611+0000 7f06f7200640 1 --2- 172.21.15.3:0/3698475364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06f81076f0 0x7f06f8107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.612+0000 7f06f7200640 1 -- 172.21.15.3:0/3698475364 >> 172.21.15.3:0/3698475364 conn(0x7f06f80fd470 msgr2=0x7f06f80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.613+0000 7f06f7200640 1 -- 172.21.15.3:0/3698475364 shutdown_connections 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.613+0000 7f06f7200640 1 -- 172.21.15.3:0/3698475364 wait complete. 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.614+0000 7f06f7200640 1 Processor -- start 2024-08-01T04:18:57.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 -- start start 2024-08-01T04:18:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06f81016f0 0x7f06f819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 0x7f06f819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06f8111b30 con 0x7f06f8102090 2024-08-01T04:18:57.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f06f81119b0 con 0x7f06f81016f0 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f7200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f06f8111cb0 con 0x7f06f81076f0 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 0x7f06f819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f6800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 0x7f06f819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41176/0 (socket says 172.21.15.3:41176) 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.615+0000 7f06f6800640 1 -- 172.21.15.3:0/3846081324 learned_addr learned my addr 172.21.15.3:0/3846081324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f5400640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f6800640 1 -- 172.21.15.3:0/3846081324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06f81016f0 msgr2=0x7f06f819a1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f6800640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06f81016f0 0x7f06f819a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f6800640 1 -- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 msgr2=0x7f06f819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f6800640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f6800640 1 -- 172.21.15.3:0/3846081324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f819f150 con 0x7f06f81076f0 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f5400640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.616+0000 7f06f6800640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 0x7f06f819ea70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005cc0 tx=0x7f06f0031460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:57.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.617+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0031850 con 0x7f06f81076f0 2024-08-01T04:18:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.617+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f819f3e0 con 0x7f06f81076f0 2024-08-01T04:18:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.617+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0004520 con 0x7f06f81076f0 2024-08-01T04:18:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.617+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f002fd80 con 0x7f06f81076f0 2024-08-01T04:18:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.617+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f81a6d20 con 0x7f06f81076f0 2024-08-01T04:18:57.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.618+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d8005290 con 0x7f06f81076f0 2024-08-01T04:18:57.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.620+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f06f00046d0 con 0x7f06f81076f0 2024-08-01T04:18:57.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.622+0000 7f06eea00640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f06dc0a2cf0 0x7f06dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:18:57.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.622+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f06f00f0a40 con 0x7f06f81076f0 2024-08-01T04:18:57.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.622+0000 7f06f5e00640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f06dc0a2cf0 0x7f06dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:18:57.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.623+0000 7f06f5e00640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f06dc0a2cf0 0x7f06dc0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f06e400a930 tx=0x7f06e4005c00 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:18:57.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.624+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f06f00bd730 con 0x7f06f81076f0 2024-08-01T04:18:57.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:57 smithi160 ceph-mon[37224]: pgmap v83: 97 pgs: 97 active+clean; 2.5 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 205 MiB/s wr, 78 op/s 2024-08-01T04:18:57.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:57 smithi163 ceph-mon[37383]: pgmap v83: 97 pgs: 97 active+clean; 2.5 GiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 205 MiB/s wr, 78 op/s 2024-08-01T04:18:58.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:57.999+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d8005580 con 0x7f06f81076f0 2024-08-01T04:18:58.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.000+0000 7f06eea00640 1 -- 172.21.15.3:0/3846081324 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11911 (secure 0 0 0) 0x7f06f00bd030 con 0x7f06f81076f0 2024-08-01T04:18:58.003 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:18:58.003 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-08-01T04:18:17:181881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:18:16.489018+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:18:58.003 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-08-01T04:18:58.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f06dc0a2cf0 msgr2=0x7f06dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f06dc0a2cf0 0x7f06dc0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f06e400a930 tx=0x7f06e4005c00 comp rx=0 tx=0).stop 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 msgr2=0x7f06f819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 0x7f06f819ea70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005cc0 tx=0x7f06f0031460 comp rx=0 tx=0).stop 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 shutdown_connections 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f06dc0a2cf0 0x7f06dc0a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06f81076f0 0x7f06f819ea70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06f8102090 0x7f06f819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 --2- 172.21.15.3:0/3846081324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06f81016f0 0x7f06f819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:18:58.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.003+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 >> 172.21.15.3:0/3846081324 conn(0x7f06f80fd470 msgr2=0x7f06f810ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:18:58.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.004+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 shutdown_connections 2024-08-01T04:18:58.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:18:58.004+0000 7f06f7200640 1 -- 172.21.15.3:0/3846081324 wait complete. 2024-08-01T04:18:58.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:58 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3846081324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:58.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:58 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3846081324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:58.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:58 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3846081324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:18:59.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:18:59.552 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:18:59 smithi003 ceph-mon[32262]: pgmap v84: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 318 MiB/s wr, 120 op/s 2024-08-01T04:18:59.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:18:59 smithi160 ceph-mon[37224]: pgmap v84: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 318 MiB/s wr, 120 op/s 2024-08-01T04:18:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:18:59 smithi163 ceph-mon[37383]: pgmap v84: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 318 MiB/s wr, 120 op/s 2024-08-01T04:18:59.906 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 -- 172.21.15.3:0/1388983242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 msgr2=0x7fe9a4112240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/1388983242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 0x7fe9a4112240 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe990009860 tx=0x7fe99002a340 comp rx=0 tx=0).stop 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 -- 172.21.15.3:0/1388983242 shutdown_connections 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/1388983242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 0x7fe9a4114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/1388983242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 0x7fe9a4112240 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/1388983242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a4076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 -- 172.21.15.3:0/1388983242 >> 172.21.15.3:0/1388983242 conn(0x7fe9a40ff780 msgr2=0x7fe9a4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:00.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 -- 172.21.15.3:0/1388983242 shutdown_connections 2024-08-01T04:19:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.730+0000 7fe9a9e00640 1 -- 172.21.15.3:0/1388983242 wait complete. 2024-08-01T04:19:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.732+0000 7fe9a9e00640 1 Processor -- start 2024-08-01T04:19:00.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.732+0000 7fe9a9e00640 1 -- start start 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a9e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a41a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a9e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 0x7fe9a41a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a9e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 0x7fe9a419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a9e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4118300 con 0x7fe9a4076450 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a9e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4118180 con 0x7fe9a4112780 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a9e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4118480 con 0x7fe9a4076d70 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a41a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a41a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37058/0 (socket says 172.21.15.3:37058) 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a2a00640 1 -- 172.21.15.3:0/3352823007 learned_addr learned my addr 172.21.15.3:0/3352823007 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.733+0000 7fe9a3400640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 0x7fe9a419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.734+0000 7fe9a2a00640 1 -- 172.21.15.3:0/3352823007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 msgr2=0x7fe9a41a4b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:19:00.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.737+0000 7fe9a2000640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 0x7fe9a41a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.738+0000 7fe9a2a00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 0x7fe9a41a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.738+0000 7fe9a2a00640 1 -- 172.21.15.3:0/3352823007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 msgr2=0x7fe9a419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.738+0000 7fe9a2a00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 0x7fe9a419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.738+0000 7fe9a2a00640 1 -- 172.21.15.3:0/3352823007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a419ef70 con 0x7fe9a4076450 2024-08-01T04:19:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.738+0000 7fe9a3400640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 0x7fe9a419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:19:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.739+0000 7fe9a2a00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a41a4630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe99400e9c0 tx=0x7fe99400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:00.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.742+0000 7fe9a8a00640 1 -- 172.21.15.3:0/3352823007 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99400ccc0 con 0x7fe9a4076450 2024-08-01T04:19:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.743+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a419f260 con 0x7fe9a4076450 2024-08-01T04:19:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.743+0000 7fe9a8a00640 1 -- 172.21.15.3:0/3352823007 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994005500 con 0x7fe9a4076450 2024-08-01T04:19:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.743+0000 7fe9a8a00640 1 -- 172.21.15.3:0/3352823007 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994010430 con 0x7fe9a4076450 2024-08-01T04:19:00.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.743+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41ab5b0 con 0x7fe9a4076450 2024-08-01T04:19:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.747+0000 7fe9ac9d1740 1 Processor -- start 2024-08-01T04:19:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.747+0000 7fe9ac9d1740 1 -- start start 2024-08-01T04:19:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.747+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdf4f0 0x561556cdf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.747+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdfe10 0x561556cea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.748+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556cecac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.748+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561556c44bf0 con 0x561556cea6d0 2024-08-01T04:19:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.748+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561556bdcb20 con 0x561556cdfe10 2024-08-01T04:19:00.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.748+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561556c40bb0 con 0x561556cdf4f0 2024-08-01T04:19:00.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.749+0000 7fe9a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556cecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.749+0000 7fe9a2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdfe10 0x561556cea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.749+0000 7fe9a2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdfe10 0x561556cea190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:36038/0 (socket says 172.21.15.3:36038) 2024-08-01T04:19:00.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.749+0000 7fe9a2a00640 1 -- 172.21.15.3:0/2931560046 learned_addr learned my addr 172.21.15.3:0/2931560046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.751+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdf4f0 0x561556cdf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.751+0000 7fe9a3400640 1 -- 172.21.15.3:0/2931560046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdfe10 msgr2=0x561556cea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.751+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdfe10 0x561556cea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.751+0000 7fe9a3400640 1 -- 172.21.15.3:0/2931560046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 msgr2=0x561556cecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.751+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556cecac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.751+0000 7fe9a3400640 1 -- 172.21.15.3:0/2931560046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561556be37f0 con 0x561556cdf4f0 2024-08-01T04:19:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.752+0000 7fe9a8a00640 1 -- 172.21.15.3:0/3352823007 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe9940046c0 con 0x7fe9a4076450 2024-08-01T04:19:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.754+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdf4f0 0x561556cdf8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9a406b190 tx=0x7fe99c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2ff6369cdec132 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.754+0000 7fe98fe00640 1 -- 172.21.15.3:0/2931560046 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c004d20 con 0x561556cdf4f0 2024-08-01T04:19:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.754+0000 7fe98fe00640 1 -- 172.21.15.3:0/2931560046 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99c00f970 con 0x561556cdf4f0 2024-08-01T04:19:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.754+0000 7fe98fe00640 1 -- 172.21.15.3:0/2931560046 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c004510 con 0x561556cdf4f0 2024-08-01T04:19:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9a8a00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe9880a2e40 0x7fe9880a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2931560046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdf4f0 msgr2=0x561556cdf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdf4f0 0x561556cdf8d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9a406b190 tx=0x7fe99c011bf0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9a8a00640 1 -- 172.21.15.3:0/3352823007 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe9940d6470 con 0x7fe9a4076450 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2931560046 shutdown_connections 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556cecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdfe10 0x561556cea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2931560046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdf4f0 0x561556cdf8d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.755+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2931560046 >> 172.21.15.3:0/2931560046 conn(0x561556cdba80 msgr2=0x561556cdbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:00.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.756+0000 7fe9a2a00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe9880a2e40 0x7fe9880a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.757+0000 7fe9a2a00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe9880a2e40 0x7fe9880a5300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x561556cdc8e0 tx=0x7fe994009f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.757+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2931560046 shutdown_connections 2024-08-01T04:19:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.757+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2931560046 wait complete. 2024-08-01T04:19:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.757+0000 7fe9ac9d1740 1 -- start start 2024-08-01T04:19:00.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 Processor -- start 2024-08-01T04:19:00.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdf4f0 0x561556ce7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556ce7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cea6d0 0x561556ce7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561556ced2a0 con 0x561556cdf4f0 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561556bffcf0 con 0x561556cea6d0 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561556ced420 con 0x561556cdfe10 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.758+0000 7fe9ac9d1740 10 client.0 _ll_register_callbacks cb 0x561556bebe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.759+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdf4f0 0x561556ce7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.759+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdf4f0 0x561556ce7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37078/0 (socket says 172.21.15.3:37078) 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.759+0000 7fe9a3400640 1 -- 172.21.15.3:0/1102608131 learned_addr learned my addr 172.21.15.3:0/1102608131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.759+0000 7fe9a3400640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cea6d0 0x561556ce7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556ce7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 -- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cea6d0 msgr2=0x561556ce7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cea6d0 0x561556ce7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 -- 172.21.15.3:0/1102608131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdf4f0 msgr2=0x561556ce7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdf4f0 0x561556ce7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 -- 172.21.15.3:0/1102608131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615569b63c0 con 0x561556cdfe10 2024-08-01T04:19:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe9a2000640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556ce7ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9a419ff50 tx=0x7fe990004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501b61e42b0350ac server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.760+0000 7fe98cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:19:00.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.761+0000 7fe98cc00640 1 -- 172.21.15.3:0/1102608131 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990037070 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.761+0000 7fe98cc00640 1 -- 172.21.15.3:0/1102608131 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe990030750 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.761+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99c00f970 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.762+0000 7fe98cc00640 1 -- 172.21.15.3:0/1102608131 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe990030340 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.762+0000 7fe9ac9d1740 10 client.25753 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.762+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561556cf8f90 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.762+0000 7fe98cc00640 1 -- 172.21.15.3:0/1102608131 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe990038040 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.762+0000 7fe98cc00640 1 -- 172.21.15.3:0/1102608131 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe990004c60 con 0x561556cdfe10 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9ac9d1740 10 client.25753 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9ac9d1740 10 client.25753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9ac9d1740 20 client.25753 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556cfa4f0 0x561556cfc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9ac9d1740 4 client.25753 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:19:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe99c00f970 con 0x561556cfa4f0 2024-08-01T04:19:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.763+0000 7fe9a3400640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556cfa4f0 0x561556cfc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.764+0000 7fe9a3400640 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556cfa4f0 0x561556cfc910 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b1a286e3842a615 server_cookie=d2d351d293f7c717 in_seq=0 out_seq=0 2024-08-01T04:19:00.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.764+0000 7fe98cc00640 10 client.25753 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:19:00.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.765+0000 7fe98cc00640 1 -- 172.21.15.3:0/1102608131 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe99c00f970 con 0x561556cfa4f0 2024-08-01T04:19:00.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.765+0000 7fe98cc00640 10 client.25753 handle_command_reply: tid=0 2024-08-01T04:19:00.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.765+0000 7fe9ac9d1740 1 client.25753 shutdown 2024-08-01T04:19:00.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.765+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 msgr2=0x561556ce7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.765+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556ce7ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9a419ff50 tx=0x7fe990004270 comp rx=0 tx=0).stop 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 shutdown_connections 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cea6d0 0x561556ce7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556cfa4f0 0x561556cfc910 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe98cc00640 0 client.25753 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556ce7ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/1102608131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdf4f0 0x561556ce7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.766+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 >> 172.21.15.3:0/1102608131 conn(0x561556cdba80 msgr2=0x561556cdac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.767+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 shutdown_connections 2024-08-01T04:19:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.767+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/1102608131 wait complete. 2024-08-01T04:19:00.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.767+0000 7fe9ac9d1740 20 client.25753 trim_cache size 0 max 16384 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 Processor -- start 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 -- start start 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556ce1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 0x561556ce5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cea6d0 0x561556ce7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe984011190 con 0x561556cdfe10 2024-08-01T04:19:00.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe98400ea60 con 0x561556cdf4f0 2024-08-01T04:19:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.793+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe98400c8b0 con 0x561556cea6d0 2024-08-01T04:19:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.794+0000 7fe9a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 0x561556ce5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.794+0000 7fe9a2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 0x561556ce5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37094/0 (socket says 172.21.15.3:37094) 2024-08-01T04:19:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.794+0000 7fe9a2000640 1 -- 172.21.15.3:0/2745924004 learned_addr learned my addr 172.21.15.3:0/2745924004 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:00.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.794+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556ce1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.794+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cea6d0 0x561556ce7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.795+0000 7fe9a2000640 1 -- 172.21.15.3:0/2745924004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cea6d0 msgr2=0x561556ce7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.795+0000 7fe9a2000640 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cea6d0 0x561556ce7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.795+0000 7fe9a2000640 1 -- 172.21.15.3:0/2745924004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 msgr2=0x561556ce1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.795+0000 7fe9a2000640 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556ce1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.795+0000 7fe9a2000640 1 -- 172.21.15.3:0/2745924004 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561556cfcf70 con 0x561556cdfe10 2024-08-01T04:19:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.795+0000 7fe9a2000640 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 0x561556ce5e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe990030b80 tx=0x7fe99002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb09f4d82936504 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.796+0000 7fe98f400640 1 -- 172.21.15.3:0/2745924004 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990052020 con 0x561556cdfe10 2024-08-01T04:19:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.796+0000 7fe98f400640 1 -- 172.21.15.3:0/2745924004 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9900054e0 con 0x561556cdfe10 2024-08-01T04:19:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.796+0000 7fe98f400640 1 -- 172.21.15.3:0/2745924004 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990037040 con 0x561556cdfe10 2024-08-01T04:19:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2745924004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 msgr2=0x561556ce5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 0x561556ce5e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe990030b80 tx=0x7fe99002a340 comp rx=0 tx=0).stop 2024-08-01T04:19:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2745924004 shutdown_connections 2024-08-01T04:19:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cea6d0 0x561556ce7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cdfe10 0x561556ce5e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2745924004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556ce1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2745924004 >> 172.21.15.3:0/2745924004 conn(0x561556cdba80 msgr2=0x561556ce95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:00.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.797+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2745924004 shutdown_connections 2024-08-01T04:19:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.798+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2745924004 wait complete. 2024-08-01T04:19:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.798+0000 7fe9ac9d1740 1 -- start start 2024-08-01T04:19:00.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.798+0000 7fe9ac9d1740 1 Processor -- start 2024-08-01T04:19:00.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556cede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556cee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556d5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe98400cf20 con 0x561556cea6d0 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe98400e390 con 0x561556cdf4f0 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe98400d5f0 con 0x561556cdfe10 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9ac9d1740 10 client.0 _ll_register_callbacks cb 0x7fe99c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556cede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556cede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:36070/0 (socket says 172.21.15.3:36070) 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.800+0000 7fe9a3400640 1 -- 172.21.15.3:0/2645407322 learned_addr learned my addr 172.21.15.3:0/2645407322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:00.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.801+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556d5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.801+0000 7fe9a2000640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556cee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.802+0000 7fe9a3400640 1 -- 172.21.15.3:0/2645407322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 msgr2=0x561556cee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.802+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556cee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.802+0000 7fe9a3400640 1 -- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 msgr2=0x561556cede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:00.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.802+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556cede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.802+0000 7fe9a3400640 1 -- 172.21.15.3:0/2645407322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561556d5d800 con 0x561556cea6d0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.803+0000 7fe9a3400640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556d5d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe99c011730 tx=0x7fe990009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc3c7a9d5c7791d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.803+0000 7fe98cc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.804+0000 7fe98cc00640 1 -- 172.21.15.3:0/2645407322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c01e040 con 0x561556cea6d0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.804+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9900054e0 con 0x561556cea6d0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.804+0000 7fe98cc00640 1 -- 172.21.15.3:0/2645407322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe99c003d90 con 0x561556cea6d0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.804+0000 7fe98cc00640 1 -- 172.21.15.3:0/2645407322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe99c005290 con 0x561556cea6d0 2024-08-01T04:19:00.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.804+0000 7fe9ac9d1740 10 client.15639 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.805+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561556d64b60 con 0x561556cea6d0 2024-08-01T04:19:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.805+0000 7fe98cc00640 1 -- 172.21.15.3:0/2645407322 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9900054e0 con 0x561556cea6d0 2024-08-01T04:19:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.805+0000 7fe98cc00640 1 -- 172.21.15.3:0/2645407322 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe99c024040 con 0x561556cea6d0 2024-08-01T04:19:00.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.806+0000 7fe9ac9d1740 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.806+0000 7fe9ac9d1740 10 client.15639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:19:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.806+0000 7fe9ac9d1740 20 client.15639 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.806+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556d650a0 0x561556d67490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.806+0000 7fe9ac9d1740 4 client.15639 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:19:00.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.806+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561556c52160 con 0x561556d650a0 2024-08-01T04:19:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.807+0000 7fe9a2000640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556d650a0 0x561556d67490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.807+0000 7fe9a2000640 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556d650a0 0x561556d67490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0df5037c69d4f57 server_cookie=6076addb9d473f24 in_seq=0 out_seq=0 2024-08-01T04:19:00.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.807+0000 7fe98cc00640 10 client.15639 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:19:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.810+0000 7fe98cc00640 1 -- 172.21.15.3:0/2645407322 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561556c52160 con 0x561556d650a0 2024-08-01T04:19:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.810+0000 7fe98cc00640 10 client.15639 handle_command_reply: tid=0 2024-08-01T04:19:00.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.810+0000 7fe9ac9d1740 1 client.15639 shutdown 2024-08-01T04:19:00.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.816+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 msgr2=0x561556d5d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556d5d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe99c011730 tx=0x7fe990009590 comp rx=0 tx=0).stop 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 shutdown_connections 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561556cea6d0 0x561556d5d190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x561556d650a0 0x561556d67490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe98cc00640 0 client.15639 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561556cdfe10 0x561556cee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 --2- 172.21.15.3:0/2645407322 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561556cdf4f0 0x561556cede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.817+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 >> 172.21.15.3:0/2645407322 conn(0x561556cdba80 msgr2=0x561556cf5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.818+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 shutdown_connections 2024-08-01T04:19:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.818+0000 7fe9ac9d1740 1 -- 172.21.15.3:0/2645407322 wait complete. 2024-08-01T04:19:00.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.818+0000 7fe9ac9d1740 20 client.15639 trim_cache size 0 max 16384 2024-08-01T04:19:00.821 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:19:00.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe9880a2e40 msgr2=0x7fe9880a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe9880a2e40 0x7fe9880a5300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x561556cdc8e0 tx=0x7fe994009f90 comp rx=0 tx=0).stop 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 msgr2=0x7fe9a41a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a41a4630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe99400e9c0 tx=0x7fe99400ee80 comp rx=0 tx=0).stop 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 shutdown_connections 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9a4112780 0x7fe9a419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe9880a2e40 0x7fe9880a5300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9a4076d70 0x7fe9a41a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 --2- 172.21.15.3:0/3352823007 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9a4076450 0x7fe9a41a4630 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:00.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.820+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 >> 172.21.15.3:0/3352823007 conn(0x7fe9a40ff780 msgr2=0x7fe9a4114540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:00.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.821+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 shutdown_connections 2024-08-01T04:19:00.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:00.821+0000 7fe9a9e00640 1 -- 172.21.15.3:0/3352823007 wait complete. 2024-08-01T04:19:01.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:01 smithi160 ceph-mon[37224]: pgmap v85: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 225 MiB/s wr, 82 op/s 2024-08-01T04:19:01.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:01 smithi163 ceph-mon[37383]: pgmap v85: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 225 MiB/s wr, 82 op/s 2024-08-01T04:19:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:01 smithi003 ceph-mon[32262]: pgmap v85: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 225 MiB/s wr, 82 op/s 2024-08-01T04:19:01.993 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:19:02.557 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.380+0000 7f133e000640 1 -- 172.21.15.3:0/3532176877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13381018b0 msgr2=0x7f1338101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.380+0000 7f133e000640 1 --2- 172.21.15.3:0/3532176877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13381018b0 0x7f1338101d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1324009930 tx=0x7f132402a420 comp rx=0 tx=0).stop 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.381+0000 7f133e000640 1 -- 172.21.15.3:0/3532176877 shutdown_connections 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.381+0000 7f133e000640 1 --2- 172.21.15.3:0/3532176877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1338102250 0x7f133810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.381+0000 7f133e000640 1 --2- 172.21.15.3:0/3532176877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13381018b0 0x7f1338101d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.381+0000 7f133e000640 1 --2- 172.21.15.3:0/3532176877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381078b0 0x7f1338107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.381+0000 7f133e000640 1 -- 172.21.15.3:0/3532176877 >> 172.21.15.3:0/3532176877 conn(0x7f13380fd610 msgr2=0x7f13380ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.382+0000 7f133e000640 1 -- 172.21.15.3:0/3532176877 shutdown_connections 2024-08-01T04:19:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.382+0000 7f133e000640 1 -- 172.21.15.3:0/3532176877 wait complete. 2024-08-01T04:19:03.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 Processor -- start 2024-08-01T04:19:03.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 -- start start 2024-08-01T04:19:03.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 0x7f1338108800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1338102250 0x7f1338108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13381078b0 0x7f1338109280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1338111d00 con 0x7f13381018b0 2024-08-01T04:19:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1338111b80 con 0x7f13381078b0 2024-08-01T04:19:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.383+0000 7f133e000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1338111e80 con 0x7f1338102250 2024-08-01T04:19:03.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.384+0000 7f1337400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 0x7f1338108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.384+0000 7f1337400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 0x7f1338108800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38864/0 (socket says 172.21.15.3:38864) 2024-08-01T04:19:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.384+0000 7f1337400640 1 -- 172.21.15.3:0/2470870359 learned_addr learned my addr 172.21.15.3:0/2470870359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.384+0000 7f1336a00640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1338102250 0x7f1338108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.384+0000 7f1337e00640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13381078b0 0x7f1338109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.384+0000 7f1337400640 1 -- 172.21.15.3:0/2470870359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1338102250 msgr2=0x7f1338108d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1337400640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1338102250 0x7f1338108d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1337400640 1 -- 172.21.15.3:0/2470870359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13381078b0 msgr2=0x7f1338109280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1337400640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13381078b0 0x7f1338109280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1337400640 1 -- 172.21.15.3:0/2470870359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13381aa9e0 con 0x7f13381018b0 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1337e00640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13381078b0 0x7f1338109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1336a00640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1338102250 0x7f1338108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f1337400640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 0x7f1338108800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f132800ea90 tx=0x7f132800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f132fe00640 1 -- 172.21.15.3:0/2470870359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132800ce40 con 0x7f13381018b0 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f132fe00640 1 -- 172.21.15.3:0/2470870359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13280035f0 con 0x7f13381018b0 2024-08-01T04:19:03.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f132fe00640 1 -- 172.21.15.3:0/2470870359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328010650 con 0x7f13381018b0 2024-08-01T04:19:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.385+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13381aacd0 con 0x7f13381018b0 2024-08-01T04:19:03.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.386+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381ab1c0 con 0x7f13381018b0 2024-08-01T04:19:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.389+0000 7f1340ac8740 1 Processor -- start 2024-08-01T04:19:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.389+0000 7f1340ac8740 1 -- start start 2024-08-01T04:19:03.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.389+0000 7f132fe00640 1 -- 172.21.15.3:0/2470870359 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f13280037a0 con 0x7f13381018b0 2024-08-01T04:19:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1340ac8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561e9b716bf0 con 0x561e9b7b1e10 2024-08-01T04:19:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1340ac8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561e9b6aeb20 con 0x561e9b7b14f0 2024-08-01T04:19:03.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1340ac8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e9b712bb0 con 0x561e9b7bc6d0 2024-08-01T04:19:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1337e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1337400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1337e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46030/0 (socket says 172.21.15.3:46030) 2024-08-01T04:19:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1337400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7bc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38876/0 (socket says 172.21.15.3:38876) 2024-08-01T04:19:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1337e00640 1 -- 172.21.15.3:0/1897921671 learned_addr learned my addr 172.21.15.3:0/1897921671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:03.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.390+0000 7f1336a00640 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f1337e00640 1 -- 172.21.15.3:0/1897921671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 msgr2=0x561e9b7beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f1337e00640 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f1337e00640 1 -- 172.21.15.3:0/1897921671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 msgr2=0x561e9b7bc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f1337e00640 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7bc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f1337e00640 1 -- 172.21.15.3:0/1897921671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e9b6b57f0 con 0x561e9b7b14f0 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f132fe00640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f131c0a2d70 0x7f131c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f1337e00640 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b18d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1330009ea0 tx=0x7f1330009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fc00ec8352fcc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.391+0000 7f132fe00640 1 -- 172.21.15.3:0/2470870359 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f13280cf8d0 con 0x7f13381018b0 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.392+0000 7f1323e00640 1 -- 172.21.15.3:0/1897921671 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330004d20 con 0x561e9b7b14f0 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.392+0000 7f1323e00640 1 -- 172.21.15.3:0/1897921671 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1330005500 con 0x561e9b7b14f0 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.392+0000 7f1323e00640 1 -- 172.21.15.3:0/1897921671 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300118d0 con 0x561e9b7b14f0 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.392+0000 7f1337400640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f131c0a2d70 0x7f131c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 -- 172.21.15.3:0/1897921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 msgr2=0x561e9b7b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b18d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1330009ea0 tx=0x7f1330009f90 comp rx=0 tx=0).stop 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 -- 172.21.15.3:0/1897921671 shutdown_connections 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1897921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b18d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1340ac8740 1 -- 172.21.15.3:0/1897921671 >> 172.21.15.3:0/1897921671 conn(0x561e9b7ada80 msgr2=0x561e9b7ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:03.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.393+0000 7f1337400640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f131c0a2d70 0x7f131c0a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13280cd090 tx=0x7f1328003c90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:03.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.394+0000 7f1340ac8740 1 -- 172.21.15.3:0/1897921671 shutdown_connections 2024-08-01T04:19:03.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.394+0000 7f1340ac8740 1 -- 172.21.15.3:0/1897921671 wait complete. 2024-08-01T04:19:03.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.394+0000 7f1340ac8740 1 -- start start 2024-08-01T04:19:03.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.394+0000 7f1340ac8740 1 Processor -- start 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7ba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561e9b7bf2a0 con 0x561e9b7b14f0 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561e9b6d1cf0 con 0x561e9b7b1e10 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e9b7bf420 con 0x561e9b7bc6d0 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1340ac8740 10 client.0 _ll_register_callbacks cb 0x561e9b6bde30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1337e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1337e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38890/0 (socket says 172.21.15.3:38890) 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.395+0000 7f1337e00640 1 -- 172.21.15.3:0/3203722120 learned_addr learned my addr 172.21.15.3:0/3203722120 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1336a00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 -- 172.21.15.3:0/3203722120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 msgr2=0x561e9b7ba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7ba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 -- 172.21.15.3:0/3203722120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 msgr2=0x561e9b7b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 -- 172.21.15.3:0/3203722120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e9b4883c0 con 0x561e9b7b14f0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1337e00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1330009760 tx=0x7f1330011470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9ffef559bb1a5f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.396+0000 7f1320c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1320c00640 1 -- 172.21.15.3:0/3203722120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330023040 con 0x561e9b7b14f0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1320c00640 1 -- 172.21.15.3:0/3203722120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1330005200 con 0x561e9b7b14f0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1330005500 con 0x561e9b7b14f0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1320c00640 1 -- 172.21.15.3:0/3203722120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1330011610 con 0x561e9b7b14f0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1340ac8740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e9b7cb010 con 0x561e9b7b14f0 2024-08-01T04:19:03.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.398+0000 7f1320c00640 1 -- 172.21.15.3:0/3203722120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1330005500 con 0x561e9b7b14f0 2024-08-01T04:19:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1320c00640 1 -- 172.21.15.3:0/3203722120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f133002a5f0 con 0x561e9b7b14f0 2024-08-01T04:19:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1340ac8740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1340ac8740 10 client.15651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:19:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1340ac8740 20 client.15651 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1340ac8740 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b7cc3c0 0x561e9b7ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1340ac8740 4 client.15651 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:19:03.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e9b724160 con 0x561e9b7cc3c0 2024-08-01T04:19:03.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.399+0000 7f1336a00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b7cc3c0 0x561e9b7ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.400+0000 7f1336a00640 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b7cc3c0 0x561e9b7ce7b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fba85729e35d509 server_cookie=e1407f251db868e2 in_seq=0 out_seq=0 2024-08-01T04:19:03.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.400+0000 7f1320c00640 10 client.15651 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:19:03.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.400+0000 7f1320c00640 1 -- 172.21.15.3:0/3203722120 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e9b724160 con 0x561e9b7cc3c0 2024-08-01T04:19:03.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1320c00640 10 client.15651 handle_command_reply: tid=0 2024-08-01T04:19:03.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 client.15651 shutdown 2024-08-01T04:19:03.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 msgr2=0x561e9b7b9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1330009760 tx=0x7f1330011470 comp rx=0 tx=0).stop 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 shutdown_connections 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b7cc3c0 0x561e9b7ce7b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7ba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 --2- 172.21.15.3:0/3203722120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1320c00640 0 client.15651 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 >> 172.21.15.3:0/3203722120 conn(0x561e9b7ada80 msgr2=0x561e9b7b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.401+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 shutdown_connections 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.402+0000 7f1340ac8740 1 -- 172.21.15.3:0/3203722120 wait complete. 2024-08-01T04:19:03.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.402+0000 7f1340ac8740 20 client.15651 trim_cache size 0 max 16384 2024-08-01T04:19:03.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 Processor -- start 2024-08-01T04:19:03.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 -- start start 2024-08-01T04:19:03.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7d3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b6760 con 0x561e9b7b1e10 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b4030 con 0x561e9b7b14f0 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1340ac8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b1e80 con 0x561e9b7bc6d0 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.416+0000 7f1336a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7d3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7d3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38900/0 (socket says 172.21.15.3:38900) 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 -- 172.21.15.3:0/1371578572 learned_addr learned my addr 172.21.15.3:0/1371578572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1337e00640 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1337e00640 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 -- 172.21.15.3:0/1371578572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 msgr2=0x561e9b7d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7d43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 -- 172.21.15.3:0/1371578572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 msgr2=0x561e9b7b9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 -- 172.21.15.3:0/1371578572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e9b7ae710 con 0x561e9b7b1e10 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1337e00640 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.417+0000 7f1336a00640 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7d3ea0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f132402a9e0 tx=0x7f1324005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2111d2f62e1b8254 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.418+0000 7f1323400640 1 -- 172.21.15.3:0/1371578572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324049070 con 0x561e9b7b1e10 2024-08-01T04:19:03.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.418+0000 7f1323400640 1 -- 172.21.15.3:0/1371578572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13240044c0 con 0x561e9b7b1e10 2024-08-01T04:19:03.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.418+0000 7f1323400640 1 -- 172.21.15.3:0/1371578572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324035040 con 0x561e9b7b1e10 2024-08-01T04:19:03.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 -- 172.21.15.3:0/1371578572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 msgr2=0x561e9b7d3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7d3ea0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f132402a9e0 tx=0x7f1324005b50 comp rx=0 tx=0).stop 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 -- 172.21.15.3:0/1371578572 shutdown_connections 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7bc6d0 0x561e9b7d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7b1e10 0x561e9b7d3ea0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 --2- 172.21.15.3:0/1371578572 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b14f0 0x561e9b7b9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 -- 172.21.15.3:0/1371578572 >> 172.21.15.3:0/1371578572 conn(0x561e9b7ada80 msgr2=0x561e9b7b9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 -- 172.21.15.3:0/1371578572 shutdown_connections 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 -- 172.21.15.3:0/1371578572 wait complete. 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 -- start start 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.419+0000 7f1340ac8740 1 Processor -- start 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7b14f0 0x561e9b7bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7bc6d0 0x561e9b834e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b24f0 con 0x561e9b7bc6d0 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b3960 con 0x561e9b7b1e10 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f13381b2bc0 con 0x561e9b7b14f0 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1340ac8740 10 client.0 _ll_register_callbacks cb 0x7f1330004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1337e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7b14f0 0x561e9b7bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1336a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1336a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7bfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46056/0 (socket says 172.21.15.3:46056) 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1337e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7b14f0 0x561e9b7bf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:60674/0 (socket says 172.21.15.3:60674) 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1336a00640 1 -- 172.21.15.3:0/2123849288 learned_addr learned my addr 172.21.15.3:0/2123849288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.420+0000 7f1337e00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7bc6d0 0x561e9b834e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1336a00640 1 -- 172.21.15.3:0/2123849288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7b14f0 msgr2=0x561e9b7bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1336a00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7b14f0 0x561e9b7bf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1336a00640 1 -- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7bc6d0 msgr2=0x561e9b834e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1336a00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7bc6d0 0x561e9b834e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1336a00640 1 -- 172.21.15.3:0/2123849288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e9b8353a0 con 0x561e9b7b1e10 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1337e00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7bc6d0 0x561e9b834e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1336a00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7bfcf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c00 tx=0x7f132402aae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72e399e5adc3087 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.421+0000 7f1320c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1320c00640 1 -- 172.21.15.3:0/2123849288 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324049040 con 0x561e9b7b1e10 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e9b7a8ff0 con 0x561e9b7b1e10 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1320c00640 1 -- 172.21.15.3:0/2123849288 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f13240315b0 con 0x561e9b7b1e10 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1320c00640 1 -- 172.21.15.3:0/2123849288 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f132405a050 con 0x561e9b7b1e10 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1340ac8740 10 client.25790 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561e9b836520 con 0x561e9b7b1e10 2024-08-01T04:19:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.422+0000 7f1320c00640 1 -- 172.21.15.3:0/2123849288 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561e9b7a8ff0 con 0x561e9b7b1e10 2024-08-01T04:19:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1320c00640 1 -- 172.21.15.3:0/2123849288 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1324055030 con 0x561e9b7b1e10 2024-08-01T04:19:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1340ac8740 10 client.25790 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1340ac8740 10 client.25790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:19:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1340ac8740 20 client.25790 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1340ac8740 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b836cd0 0x561e9b8390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1340ac8740 4 client.25790 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e9b71e670 con 0x561e9b836cd0 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.423+0000 7f1337e00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b836cd0 0x561e9b8390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1320c00640 10 client.25790 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1337e00640 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b836cd0 0x561e9b8390f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e83c3759f1cf799 server_cookie=d6cc363d96cc9318 in_seq=0 out_seq=0 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1320c00640 1 -- 172.21.15.3:0/2123849288 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e9b71e670 con 0x561e9b836cd0 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1320c00640 10 client.25790 handle_command_reply: tid=0 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1340ac8740 1 client.25790 shutdown 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 msgr2=0x561e9b7bfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.424+0000 7f1340ac8740 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7bfcf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c00 tx=0x7f132402aae0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 shutdown_connections 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e9b836cd0 0x561e9b8390f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e9b7bc6d0 0x561e9b834e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e9b7b1e10 0x561e9b7bfcf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 --2- 172.21.15.3:0/2123849288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e9b7b14f0 0x561e9b7bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 >> 172.21.15.3:0/2123849288 conn(0x561e9b7ada80 msgr2=0x561e9b7acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1320c00640 0 client.25790 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 shutdown_connections 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 1 -- 172.21.15.3:0/2123849288 wait complete. 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.425+0000 7f1340ac8740 20 client.25790 trim_cache size 0 max 16384 2024-08-01T04:19:03.429 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f131c0a2d70 msgr2=0x7f131c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f131c0a2d70 0x7f131c0a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13280cd090 tx=0x7f1328003c90 comp rx=0 tx=0).stop 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 msgr2=0x7f1338108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 0x7f1338108800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f132800ea90 tx=0x7f132800ef50 comp rx=0 tx=0).stop 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 shutdown_connections 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13381078b0 0x7f1338109280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f131c0a2d70 0x7f131c0a5230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1338102250 0x7f1338108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 --2- 172.21.15.3:0/2470870359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13381018b0 0x7f1338108800 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 >> 172.21.15.3:0/2470870359 conn(0x7f13380fd610 msgr2=0x7f133810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:03.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 shutdown_connections 2024-08-01T04:19:03.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:03.426+0000 7f133e000640 1 -- 172.21.15.3:0/2470870359 wait complete. 2024-08-01T04:19:03.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:03 smithi003 ceph-mon[32262]: pgmap v86: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 302 MiB/s wr, 129 op/s 2024-08-01T04:19:03.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:03 smithi160 ceph-mon[37224]: pgmap v86: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 302 MiB/s wr, 129 op/s 2024-08-01T04:19:03.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:03 smithi163 ceph-mon[37383]: pgmap v86: 97 pgs: 97 active+clean; 4.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 302 MiB/s wr, 129 op/s 2024-08-01T04:19:04.648 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:19:05.199 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:19:05.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:05 smithi003 ceph-mon[32262]: pgmap v87: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 223 MiB/s wr, 96 op/s 2024-08-01T04:19:05.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:05 smithi160 ceph-mon[37224]: pgmap v87: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 223 MiB/s wr, 96 op/s 2024-08-01T04:19:05.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:05 smithi163 ceph-mon[37383]: pgmap v87: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 223 MiB/s wr, 96 op/s 2024-08-01T04:19:05.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.955+0000 7fab23e00640 1 -- 172.21.15.3:0/455647536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab2410bcf0 msgr2=0x7fab2410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.955+0000 7fab23e00640 1 --2- 172.21.15.3:0/455647536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab2410bcf0 0x7fab2410e190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009ca0 tx=0x7fab1c02a970 comp rx=0 tx=0).stop 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.956+0000 7fab23e00640 1 -- 172.21.15.3:0/455647536 shutdown_connections 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.956+0000 7fab23e00640 1 --2- 172.21.15.3:0/455647536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab2410bcf0 0x7fab2410e190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.956+0000 7fab23e00640 1 --2- 172.21.15.3:0/455647536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241031e0 0x7fab2410b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.956+0000 7fab23e00640 1 --2- 172.21.15.3:0/455647536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab241028c0 0x7fab24102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.956+0000 7fab23e00640 1 -- 172.21.15.3:0/455647536 >> 172.21.15.3:0/455647536 conn(0x7fab240fc340 msgr2=0x7fab240fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:05.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.958+0000 7fab23e00640 1 -- 172.21.15.3:0/455647536 shutdown_connections 2024-08-01T04:19:05.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.959+0000 7fab23e00640 1 -- 172.21.15.3:0/455647536 wait complete. 2024-08-01T04:19:05.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.960+0000 7fab23e00640 1 Processor -- start 2024-08-01T04:19:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.962+0000 7fab23e00640 1 -- start start 2024-08-01T04:19:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.963+0000 7fab23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241028c0 0x7fab2419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.963+0000 7fab23e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab241031e0 0x7fab2419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.963+0000 7fab23e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2410bcf0 0x7fab2419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.963+0000 7fab23e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab24113e20 con 0x7fab241028c0 2024-08-01T04:19:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.963+0000 7fab23e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab24113ca0 con 0x7fab241031e0 2024-08-01T04:19:05.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.963+0000 7fab23e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab24113fa0 con 0x7fab2410bcf0 2024-08-01T04:19:05.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.964+0000 7fab22a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241028c0 0x7fab2419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.964+0000 7fab22000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab241031e0 0x7fab2419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.964+0000 7fab22000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab241031e0 0x7fab2419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46084/0 (socket says 172.21.15.3:46084) 2024-08-01T04:19:05.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.964+0000 7fab22000640 1 -- 172.21.15.3:0/1006367524 learned_addr learned my addr 172.21.15.3:0/1006367524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.964+0000 7fab23400640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2410bcf0 0x7fab2419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.965+0000 7fab22a00640 1 -- 172.21.15.3:0/1006367524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2410bcf0 msgr2=0x7fab2419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.965+0000 7fab22a00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2410bcf0 0x7fab2419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.965+0000 7fab22a00640 1 -- 172.21.15.3:0/1006367524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab241031e0 msgr2=0x7fab2419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.965+0000 7fab22a00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab241031e0 0x7fab2419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.965+0000 7fab22a00640 1 -- 172.21.15.3:0/1006367524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2419f290 con 0x7fab241028c0 2024-08-01T04:19:05.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.966+0000 7fab22a00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241028c0 0x7fab2419a2e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b750 tx=0x7fab1400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:05.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.967+0000 7fab1b400640 1 -- 172.21.15.3:0/1006367524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14019070 con 0x7fab241028c0 2024-08-01T04:19:05.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.967+0000 7fab1b400640 1 -- 172.21.15.3:0/1006367524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1400cc00 con 0x7fab241028c0 2024-08-01T04:19:05.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.967+0000 7fab1b400640 1 -- 172.21.15.3:0/1006367524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14012340 con 0x7fab241028c0 2024-08-01T04:19:05.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.967+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2419f580 con 0x7fab241028c0 2024-08-01T04:19:05.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.967+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2410ecd0 con 0x7fab241028c0 2024-08-01T04:19:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.972+0000 7fab2c1e9740 1 Processor -- start 2024-08-01T04:19:05.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.972+0000 7fab1b400640 1 -- 172.21.15.3:0/1006367524 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fab1401c070 con 0x7fab241028c0 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 -- start start 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb78ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb83380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb838c0 0x56366bb85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56366baddbf0 con 0x56366bb786e0 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56366ba75b20 con 0x56366bb79000 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.973+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56366bad9bb0 con 0x56366bb838c0 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.974+0000 7fab22a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb83380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.974+0000 7fab22a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb83380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46088/0 (socket says 172.21.15.3:46088) 2024-08-01T04:19:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.974+0000 7fab22a00640 1 -- 172.21.15.3:0/578740345 learned_addr learned my addr 172.21.15.3:0/578740345 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:05.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab1b400640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab080a2e40 0x7fab080a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab22a00640 1 -- 172.21.15.3:0/578740345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb838c0 msgr2=0x56366bb85cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:19:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab22a00640 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb838c0 0x56366bb85cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab22a00640 1 -- 172.21.15.3:0/578740345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 msgr2=0x56366bb78ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:05.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab22a00640 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb78ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab22a00640 1 -- 172.21.15.3:0/578740345 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366ba7c7f0 con 0x56366bb79000 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.975+0000 7fab22a00640 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb83380 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab140cba00 tx=0x7fab140969f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba166a4042dc8437 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.976+0000 7fab0f400640 1 -- 172.21.15.3:0/578740345 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1409fe60 con 0x56366bb79000 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.976+0000 7fab0f400640 1 -- 172.21.15.3:0/578740345 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab140a47f0 con 0x56366bb79000 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.976+0000 7fab0f400640 1 -- 172.21.15.3:0/578740345 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140a4be0 con 0x56366bb79000 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.976+0000 7fab23400640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab080a2e40 0x7fab080a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.977+0000 7fab2c1e9740 1 -- 172.21.15.3:0/578740345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 msgr2=0x56366bb83380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.977+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb83380 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab140cba00 tx=0x7fab140969f0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab1b400640 1 -- 172.21.15.3:0/1006367524 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fab140ce300 con 0x7fab241028c0 2024-08-01T04:19:05.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab23400640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab080a2e40 0x7fab080a5300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009430 tx=0x7fab1c009360 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab2c1e9740 1 -- 172.21.15.3:0/578740345 shutdown_connections 2024-08-01T04:19:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb838c0 0x56366bb85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb83380 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/578740345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb78ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.978+0000 7fab2c1e9740 1 -- 172.21.15.3:0/578740345 >> 172.21.15.3:0/578740345 conn(0x56366bb74c70 msgr2=0x56366bb75050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:05.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.979+0000 7fab2c1e9740 1 -- 172.21.15.3:0/578740345 shutdown_connections 2024-08-01T04:19:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.979+0000 7fab2c1e9740 1 -- 172.21.15.3:0/578740345 wait complete. 2024-08-01T04:19:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.979+0000 7fab2c1e9740 1 -- start start 2024-08-01T04:19:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.980+0000 7fab2c1e9740 1 Processor -- start 2024-08-01T04:19:05.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.980+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.980+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb80d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.980+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb81250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56366bb974a0 con 0x56366bb838c0 2024-08-01T04:19:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56366ba98cf0 con 0x56366bb79000 2024-08-01T04:19:05.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56366bb97620 con 0x56366bb786e0 2024-08-01T04:19:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab2c1e9740 10 client.0 _ll_register_callbacks cb 0x56366ba84e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:05.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab22000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb80d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab22a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab22000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb80d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46096/0 (socket says 172.21.15.3:46096) 2024-08-01T04:19:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab22000640 1 -- 172.21.15.3:0/108537672 learned_addr learned my addr 172.21.15.3:0/108537672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab23400640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb81250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab22000640 1 -- 172.21.15.3:0/108537672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 msgr2=0x56366bb807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.981+0000 7fab22000640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.982+0000 7fab22000640 1 -- 172.21.15.3:0/108537672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 msgr2=0x56366bb81250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.982+0000 7fab22000640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb81250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.982+0000 7fab22000640 1 -- 172.21.15.3:0/108537672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366b84f3c0 con 0x56366bb79000 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.982+0000 7fab22a00640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.983+0000 7fab22000640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb80d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e4b0 tx=0x7fab1000e970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45392cc0c46ebebf server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.983+0000 7fab03e00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.983+0000 7fab03e00640 1 -- 172.21.15.3:0/108537672 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10017070 con 0x56366bb79000 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.983+0000 7fab03e00640 1 -- 172.21.15.3:0/108537672 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fab10013680 con 0x56366bb79000 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.983+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56366baeb160 con 0x56366bb79000 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.985+0000 7fab03e00640 1 -- 172.21.15.3:0/108537672 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab10021050 con 0x56366bb79000 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.985+0000 7fab2c1e9740 10 client.25802 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.985+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56366bba3260 con 0x56366bb79000 2024-08-01T04:19:05.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.985+0000 7fab03e00640 1 -- 172.21.15.3:0/108537672 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab10013d60 con 0x56366bb79000 2024-08-01T04:19:05.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab03e00640 1 -- 172.21.15.3:0/108537672 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab1001cc10 con 0x56366bb79000 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab2c1e9740 10 client.25802 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab2c1e9740 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab2c1e9740 20 client.25802 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bba4810 0x56366bba6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab2c1e9740 4 client.25802 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.986+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366baeb160 con 0x56366bba4810 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.987+0000 7fab22a00640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bba4810 0x56366bba6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:05.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.988+0000 7fab22a00640 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bba4810 0x56366bba6c00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc946277714b7243 server_cookie=734478a25b87c4f4 in_seq=0 out_seq=0 2024-08-01T04:19:05.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.988+0000 7fab03e00640 10 client.25802 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:05.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.989+0000 7fab03e00640 1 -- 172.21.15.3:0/108537672 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56366baeb160 con 0x56366bba4810 2024-08-01T04:19:05.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.989+0000 7fab03e00640 10 client.25802 handle_command_reply: tid=0 2024-08-01T04:19:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.989+0000 7fab2c1e9740 1 client.25802 shutdown 2024-08-01T04:19:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 msgr2=0x56366bb80d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb80d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e4b0 tx=0x7fab1000e970 comp rx=0 tx=0).stop 2024-08-01T04:19:05.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 shutdown_connections 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bba4810 0x56366bba6c00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb81250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb80d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/108537672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 >> 172.21.15.3:0/108537672 conn(0x56366bb74c70 msgr2=0x56366bb7a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.990+0000 7fab03e00640 0 client.25802 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.991+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 shutdown_connections 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.991+0000 7fab2c1e9740 1 -- 172.21.15.3:0/108537672 wait complete. 2024-08-01T04:19:05.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:05.991+0000 7fab2c1e9740 20 client.25802 trim_cache size 0 max 16384 2024-08-01T04:19:06.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.022+0000 7fab2c1e9740 1 Processor -- start 2024-08-01T04:19:06.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.022+0000 7fab2c1e9740 1 -- start start 2024-08-01T04:19:06.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.023+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb7b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.023+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb79000 0x56366bb810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 0x56366bb73b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab24072710 con 0x56366bb786e0 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406ffe0 con 0x56366bb838c0 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab2c1e9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406de30 con 0x56366bb79000 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab23400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 0x56366bb73b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab23400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 0x56366bb73b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46110/0 (socket says 172.21.15.3:46110) 2024-08-01T04:19:06.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.024+0000 7fab23400640 1 -- 172.21.15.3:0/2288800130 learned_addr learned my addr 172.21.15.3:0/2288800130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.025+0000 7fab22000640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb79000 0x56366bb810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.025+0000 7fab23400640 1 -- 172.21.15.3:0/2288800130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb79000 msgr2=0x56366bb810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.025+0000 7fab22a00640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb7b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.026+0000 7fab23400640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb79000 0x56366bb810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.026+0000 7fab23400640 1 -- 172.21.15.3:0/2288800130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 msgr2=0x56366bb7b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.026+0000 7fab23400640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb7b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.026+0000 7fab23400640 1 -- 172.21.15.3:0/2288800130 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366bb85ff0 con 0x56366bb838c0 2024-08-01T04:19:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.026+0000 7fab22a00640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb7b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:19:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.026+0000 7fab22000640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb79000 0x56366bb810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:19:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.027+0000 7fab23400640 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 0x56366bb73b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab1c02f6e0 tx=0x7fab1c03c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d9bbc91c097159 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.027+0000 7fab0ea00640 1 -- 172.21.15.3:0/2288800130 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c034040 con 0x56366bb838c0 2024-08-01T04:19:06.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.027+0000 7fab0ea00640 1 -- 172.21.15.3:0/2288800130 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c051070 con 0x56366bb838c0 2024-08-01T04:19:06.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.027+0000 7fab0ea00640 1 -- 172.21.15.3:0/2288800130 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c004510 con 0x56366bb838c0 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 -- 172.21.15.3:0/2288800130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 msgr2=0x56366bb73b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 0x56366bb73b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab1c02f6e0 tx=0x7fab1c03c000 comp rx=0 tx=0).stop 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 -- 172.21.15.3:0/2288800130 shutdown_connections 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb838c0 0x56366bb73b90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb79000 0x56366bb810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/2288800130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb786e0 0x56366bb7b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 -- 172.21.15.3:0/2288800130 >> 172.21.15.3:0/2288800130 conn(0x56366bb74c70 msgr2=0x56366bb73480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:06.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 -- 172.21.15.3:0/2288800130 shutdown_connections 2024-08-01T04:19:06.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.029+0000 7fab2c1e9740 1 -- 172.21.15.3:0/2288800130 wait complete. 2024-08-01T04:19:06.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.030+0000 7fab2c1e9740 1 -- start start 2024-08-01T04:19:06.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.031+0000 7fab2c1e9740 1 Processor -- start 2024-08-01T04:19:06.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.032+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb9c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.032+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb9cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.032+0000 7fab22a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb9c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.032+0000 7fab22a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb9c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:60690/0 (socket says 172.21.15.3:60690) 2024-08-01T04:19:06.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.033+0000 7fab2c1e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb7d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.033+0000 7fab23400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb7d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.033+0000 7fab23400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb7d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38978/0 (socket says 172.21.15.3:38978) 2024-08-01T04:19:06.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab22000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb9cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab23400640 1 -- 172.21.15.3:0/338171324 learned_addr learned my addr 172.21.15.3:0/338171324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:06.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406e4a0 con 0x56366bb838c0 2024-08-01T04:19:06.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406f910 con 0x56366bb79000 2024-08-01T04:19:06.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406eb70 con 0x56366bb786e0 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab22a00640 1 -- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 msgr2=0x56366bb9cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab22a00640 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb9cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab22a00640 1 -- 172.21.15.3:0/338171324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 msgr2=0x56366bb7d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab22a00640 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb7d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab22a00640 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366bb7d5e0 con 0x56366bb786e0 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab2c1e9740 10 client.0 _ll_register_callbacks cb 0x7fab10009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.034+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c051070 con 0x56366bb786e0 2024-08-01T04:19:06.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.035+0000 7fab22a00640 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb9c8a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x56366bb75ad0 tx=0x7fab14096900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2075fd89c313811 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:06.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.035+0000 7fab03e00640 10 client.25786 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.036+0000 7fab03e00640 1 -- 172.21.15.3:0/338171324 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14083e60 con 0x56366bb786e0 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.036+0000 7fab03e00640 1 -- 172.21.15.3:0/338171324 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fab14096f90 con 0x56366bb786e0 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.036+0000 7fab03e00640 1 -- 172.21.15.3:0/338171324 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fab1409fe60 con 0x56366bb786e0 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.037+0000 7fab2c1e9740 10 client.25786 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.037+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56366bc0ec20 con 0x56366bb786e0 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab03e00640 1 -- 172.21.15.3:0/338171324 <== mon.1 v2:172.21.15.163:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fab140cbc30 con 0x56366bb786e0 2024-08-01T04:19:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab2c1e9740 10 client.25786 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab2c1e9740 10 client.25786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:19:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab2c1e9740 20 client.25786 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bc0f330 0x56366bc11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab2c1e9740 4 client.25786 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:19:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56366bae5670 con 0x56366bc0f330 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.038+0000 7fab22000640 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bc0f330 0x56366bc11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.039+0000 7fab22000640 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bc0f330 0x56366bc11750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6238728b47182d0 server_cookie=24015c581aa661ab in_seq=0 out_seq=0 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.039+0000 7fab03e00640 10 client.25786 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.039+0000 7fab03e00640 1 -- 172.21.15.3:0/338171324 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56366bae5670 con 0x56366bc0f330 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.039+0000 7fab03e00640 10 client.25786 handle_command_reply: tid=0 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.039+0000 7fab2c1e9740 1 client.25786 shutdown 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 msgr2=0x56366bb9c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb9c8a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x56366bb75ad0 tx=0x7fab14096900 comp rx=0 tx=0).stop 2024-08-01T04:19:06.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 shutdown_connections 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x56366bc0f330 0x56366bc11750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab03e00640 0 client.25786 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56366bb838c0 0x56366bb7d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56366bb79000 0x56366bb9cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 --2- 172.21.15.3:0/338171324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56366bb786e0 0x56366bb9c8a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 >> 172.21.15.3:0/338171324 conn(0x56366bb74c70 msgr2=0x56366bb73ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 shutdown_connections 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.040+0000 7fab2c1e9740 1 -- 172.21.15.3:0/338171324 wait complete. 2024-08-01T04:19:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.041+0000 7fab2c1e9740 20 client.25786 trim_cache size 0 max 16384 2024-08-01T04:19:06.045 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:19:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.042+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab080a2e40 msgr2=0x7fab080a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.042+0000 7fab23e00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab080a2e40 0x7fab080a5300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009430 tx=0x7fab1c009360 comp rx=0 tx=0).stop 2024-08-01T04:19:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.042+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241028c0 msgr2=0x7fab2419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.042+0000 7fab23e00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241028c0 0x7fab2419a2e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b750 tx=0x7fab1400bc10 comp rx=0 tx=0).stop 2024-08-01T04:19:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.043+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 shutdown_connections 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.043+0000 7fab23e00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fab080a2e40 0x7fab080a5300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.043+0000 7fab23e00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2410bcf0 0x7fab2419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.043+0000 7fab23e00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fab241031e0 0x7fab2419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.043+0000 7fab23e00640 1 --2- 172.21.15.3:0/1006367524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab241028c0 0x7fab2419a2e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.043+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 >> 172.21.15.3:0/1006367524 conn(0x7fab240fc340 msgr2=0x7fab240fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.044+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 shutdown_connections 2024-08-01T04:19:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:06.044+0000 7fab23e00640 1 -- 172.21.15.3:0/1006367524 wait complete. 2024-08-01T04:19:07.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:07 smithi003 ceph-mon[32262]: pgmap v88: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 210 MiB/s wr, 93 op/s 2024-08-01T04:19:07.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:07 smithi160 ceph-mon[37224]: pgmap v88: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 210 MiB/s wr, 93 op/s 2024-08-01T04:19:07.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:07 smithi163 ceph-mon[37383]: pgmap v88: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 210 MiB/s wr, 93 op/s 2024-08-01T04:19:08.244 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:19:08.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:19:08.952 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:19:09.382 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:09 smithi003 ceph-mon[32262]: pgmap v89: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 296 MiB/s wr, 127 op/s 2024-08-01T04:19:09.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:09 smithi160 ceph-mon[37224]: pgmap v89: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 296 MiB/s wr, 127 op/s 2024-08-01T04:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.735+0000 7f6ed6200640 1 -- 172.21.15.3:0/3905698095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 msgr2=0x7f6ed0103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.735+0000 7f6ed6200640 1 --2- 172.21.15.3:0/3905698095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 0x7f6ed0103d60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009900 tx=0x7f6ebc02a410 comp rx=0 tx=0).stop 2024-08-01T04:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 -- 172.21.15.3:0/3905698095 shutdown_connections 2024-08-01T04:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 --2- 172.21.15.3:0/3905698095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed01042a0 0x7f6ed01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 --2- 172.21.15.3:0/3905698095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 0x7f6ed0103d60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 --2- 172.21.15.3:0/3905698095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed0109900 0x7f6ed0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 -- 172.21.15.3:0/3905698095 >> 172.21.15.3:0/3905698095 conn(0x7f6ed00ff640 msgr2=0x7f6ed0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 -- 172.21.15.3:0/3905698095 shutdown_connections 2024-08-01T04:19:09.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.736+0000 7f6ed6200640 1 -- 172.21.15.3:0/3905698095 wait complete. 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 Processor -- start 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 -- start start 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 0x7f6ed0110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 0x7f6ed010d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0109900 0x7f6ed010dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0113d70 con 0x7f6ed0109900 2024-08-01T04:19:09.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0113bf0 con 0x7f6ed0103900 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.737+0000 7f6ed6200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0113ef0 con 0x7f6ed01042a0 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 0x7f6ed010d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 0x7f6ed010d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:60712/0 (socket says 172.21.15.3:60712) 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 -- 172.21.15.3:0/862727139 learned_addr learned my addr 172.21.15.3:0/862727139 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 0x7f6ed0110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ed5800640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0109900 0x7f6ed010dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 -- 172.21.15.3:0/862727139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 msgr2=0x7f6ed0110650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 0x7f6ed0110650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 -- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0109900 msgr2=0x7f6ed010dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0109900 0x7f6ed010dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ecfe00640 1 -- 172.21.15.3:0/862727139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed010e4d0 con 0x7f6ed01042a0 2024-08-01T04:19:09.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.738+0000 7f6ed5800640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0109900 0x7f6ed010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.739+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 0x7f6ed010d700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc02aa00 tx=0x7f6ebc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.739+0000 7f6ecd600640 1 -- 172.21.15.3:0/862727139 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc03f070 con 0x7f6ed01042a0 2024-08-01T04:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.739+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed01aab10 con 0x7f6ed01042a0 2024-08-01T04:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.739+0000 7f6ecd600640 1 -- 172.21.15.3:0/862727139 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc031c30 con 0x7f6ed01042a0 2024-08-01T04:19:09.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.739+0000 7f6ecd600640 1 -- 172.21.15.3:0/862727139 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc035030 con 0x7f6ed01042a0 2024-08-01T04:19:09.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.739+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01ab0a0 con 0x7f6ed01042a0 2024-08-01T04:19:09.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.742+0000 7f6ecd600640 1 -- 172.21.15.3:0/862727139 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6ebc042070 con 0x7f6ed01042a0 2024-08-01T04:19:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6ecd600640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6eb80a2cf0 0x7f6eb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6eda46a740 1 Processor -- start 2024-08-01T04:19:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6ecd600640 1 -- 172.21.15.3:0/862727139 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6ebc0efbf0 con 0x7f6ed01042a0 2024-08-01T04:19:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6eda46a740 1 -- start start 2024-08-01T04:19:09.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6eb80a2cf0 0x7f6eb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.744+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6eda46a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8ec4dd7e0 con 0x55f8ec5c2650 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6eda46a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f8ec528ac0 con 0x55f8ec5cd830 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6eda46a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f8ec4c08c0 con 0x55f8ec5c2f70 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:60724/0 (socket says 172.21.15.3:60724) 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46140/0 (socket says 172.21.15.3:46140) 2024-08-01T04:19:09.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 -- 172.21.15.3:0/1635538979 learned_addr learned my addr 172.21.15.3:0/1635538979 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 -- 172.21.15.3:0/1635538979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 msgr2=0x55f8ec5cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 -- 172.21.15.3:0/1635538979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 msgr2=0x55f8ec5c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ecfe00640 1 -- 172.21.15.3:0/1635538979 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ec4c7760 con 0x55f8ec5c2f70 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.745+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.746+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6eb80a2cf0 0x7f6eb80a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009cf0 tx=0x7f6ec4009830 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:09.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.746+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cd2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0b8a40 tx=0x7f6ebc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79c6dea3f76cd19 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.746+0000 7f6ec2000640 1 -- 172.21.15.3:0/1635538979 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc049050 con 0x55f8ec5c2f70 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.746+0000 7f6ec2000640 1 -- 172.21.15.3:0/1635538979 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc0bc070 con 0x55f8ec5c2f70 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.746+0000 7f6ec2000640 1 -- 172.21.15.3:0/1635538979 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b86e0 con 0x55f8ec5c2f70 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.747+0000 7f6eda46a740 1 -- 172.21.15.3:0/1635538979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 msgr2=0x55f8ec5cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.747+0000 7f6eda46a740 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cd2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0b8a40 tx=0x7f6ebc009260 comp rx=0 tx=0).stop 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 -- 172.21.15.3:0/1635538979 shutdown_connections 2024-08-01T04:19:09.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cd2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 --2- 172.21.15.3:0/1635538979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 -- 172.21.15.3:0/1635538979 >> 172.21.15.3:0/1635538979 conn(0x55f8ec5bebe0 msgr2=0x55f8ec5befc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 -- 172.21.15.3:0/1635538979 shutdown_connections 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 -- 172.21.15.3:0/1635538979 wait complete. 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.748+0000 7f6eda46a740 1 -- start start 2024-08-01T04:19:09.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.749+0000 7f6eda46a740 1 Processor -- start 2024-08-01T04:19:09.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.750+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.750+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6eda46a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8ec4e3ae0 con 0x55f8ec5c2f70 2024-08-01T04:19:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6eda46a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f8ec524a80 con 0x55f8ec5c2650 2024-08-01T04:19:09.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6eda46a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f8ec5d01c0 con 0x55f8ec5cd830 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6ed4e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6ed4e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46150/0 (socket says 172.21.15.3:46150) 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6ed4e00640 1 -- 172.21.15.3:0/416778207 learned_addr learned my addr 172.21.15.3:0/416778207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6ed5800640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.751+0000 7f6eda46a740 10 client.0 _ll_register_callbacks cb 0x55f8ec4e5190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ed5800640 1 -- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 msgr2=0x55f8ec5ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:09.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ed5800640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ed5800640 1 -- 172.21.15.3:0/416778207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2f70 msgr2=0x55f8ec5cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ed5800640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ed5800640 1 -- 172.21.15.3:0/416778207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ec29a6a0 con 0x55f8ec5cd830 2024-08-01T04:19:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ecfe00640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:19:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6ed5800640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cb190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800faf0 tx=0x7f6ec8007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcbacec4e5d206d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:09.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6eb6a00640 1 -- 172.21.15.3:0/416778207 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8010420 con 0x55f8ec5cd830 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.752+0000 7f6eb6a00640 1 -- 172.21.15.3:0/416778207 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6ec80105c0 con 0x55f8ec5cd830 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.753+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ebc0bc070 con 0x55f8ec5cd830 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.753+0000 7f6eb6a00640 1 -- 172.21.15.3:0/416778207 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6ec80107c0 con 0x55f8ec5cd830 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.753+0000 7f6eda46a740 10 client.25798 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.753+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8ec5dbca0 con 0x55f8ec5cd830 2024-08-01T04:19:09.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eb6a00640 1 -- 172.21.15.3:0/416778207 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ebc0bc070 con 0x55f8ec5cd830 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eb6a00640 1 -- 172.21.15.3:0/416778207 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6ec8018a10 con 0x55f8ec5cd830 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eda46a740 10 client.25798 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eda46a740 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eda46a740 20 client.25798 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eda46a740 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec5dd250 0x55f8ec5df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eda46a740 4 client.25798 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8ec536030 con 0x55f8ec5dd250 2024-08-01T04:19:09.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.754+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec5dd250 0x55f8ec5df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.755+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec5dd250 0x55f8ec5df6b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe1f70398b937fc server_cookie=933fd432ba0bd254 in_seq=0 out_seq=0 2024-08-01T04:19:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.755+0000 7f6eb6a00640 10 client.25798 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eb6a00640 1 -- 172.21.15.3:0/416778207 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f8ec536030 con 0x55f8ec5dd250 2024-08-01T04:19:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eb6a00640 10 client.25798 handle_command_reply: tid=0 2024-08-01T04:19:09.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 client.25798 shutdown 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5cd830 msgr2=0x55f8ec5cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cb190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800faf0 tx=0x7f6ec8007f70 comp rx=0 tx=0).stop 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 shutdown_connections 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec5dd250 0x55f8ec5df6b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5cd830 0x55f8ec5cb190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eb6a00640 0 client.25798 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 --2- 172.21.15.3:0/416778207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.756+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 >> 172.21.15.3:0/416778207 conn(0x55f8ec5bebe0 msgr2=0x55f8ec5c4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:09.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.757+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 shutdown_connections 2024-08-01T04:19:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.757+0000 7f6eda46a740 1 -- 172.21.15.3:0/416778207 wait complete. 2024-08-01T04:19:09.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.757+0000 7f6eda46a740 20 client.25798 trim_cache size 0 max 16384 2024-08-01T04:19:09.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:09 smithi163 ceph-mon[37383]: pgmap v89: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 296 MiB/s wr, 127 op/s 2024-08-01T04:19:09.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.794+0000 7f6eda46a740 1 Processor -- start 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.794+0000 7f6eda46a740 1 -- start start 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.795+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.795+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.795+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 0x55f8ec5e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.795+0000 7f6eda46a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01b6480 con 0x55f8ec5cd830 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.795+0000 7f6eda46a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01b3d50 con 0x55f8ec5c2650 2024-08-01T04:19:09.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.795+0000 7f6eda46a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01b1ba0 con 0x55f8ec5c2f70 2024-08-01T04:19:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.796+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.796+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 0x55f8ec5e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.796+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 0x55f8ec5e54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39022/0 (socket says 172.21.15.3:39022) 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.796+0000 7f6ed5800640 1 -- 172.21.15.3:0/770271242 learned_addr learned my addr 172.21.15.3:0/770271242 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.796+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.797+0000 7f6ed5800640 1 -- 172.21.15.3:0/770271242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 msgr2=0x55f8ec5e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.797+0000 7f6ed5800640 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.797+0000 7f6ed5800640 1 -- 172.21.15.3:0/770271242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 msgr2=0x55f8ec5cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.797+0000 7f6ed5800640 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.797+0000 7f6ed5800640 1 -- 172.21.15.3:0/770271242 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ec5cff60 con 0x55f8ec5cd830 2024-08-01T04:19:09.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.798+0000 7f6ed5800640 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 0x55f8ec5e54c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8002230 tx=0x7f6ec800f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff28c0c61c7124bf server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:09.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.799+0000 7f6ec1600640 1 -- 172.21.15.3:0/770271242 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8026040 con 0x55f8ec5cd830 2024-08-01T04:19:09.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.799+0000 7f6ec1600640 1 -- 172.21.15.3:0/770271242 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec8010e40 con 0x55f8ec5cd830 2024-08-01T04:19:09.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.799+0000 7f6ec1600640 1 -- 172.21.15.3:0/770271242 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8026040 con 0x55f8ec5cd830 2024-08-01T04:19:09.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.800+0000 7f6eda46a740 1 -- 172.21.15.3:0/770271242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 msgr2=0x55f8ec5e54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.800+0000 7f6eda46a740 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 0x55f8ec5e54c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8002230 tx=0x7f6ec800f760 comp rx=0 tx=0).stop 2024-08-01T04:19:09.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.801+0000 7f6eda46a740 1 -- 172.21.15.3:0/770271242 shutdown_connections 2024-08-01T04:19:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.801+0000 7f6eda46a740 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5cd830 0x55f8ec5e54c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.802+0000 7f6eda46a740 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.802+0000 7f6eda46a740 1 --2- 172.21.15.3:0/770271242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5c2650 0x55f8ec5cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.802+0000 7f6eda46a740 1 -- 172.21.15.3:0/770271242 >> 172.21.15.3:0/770271242 conn(0x55f8ec5bebe0 msgr2=0x55f8ec5cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:09.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.802+0000 7f6eda46a740 1 -- 172.21.15.3:0/770271242 shutdown_connections 2024-08-01T04:19:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.802+0000 7f6eda46a740 1 -- 172.21.15.3:0/770271242 wait complete. 2024-08-01T04:19:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.803+0000 7f6eda46a740 1 -- start start 2024-08-01T04:19:09.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.803+0000 7f6eda46a740 1 Processor -- start 2024-08-01T04:19:09.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec646f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01b2210 con 0x55f8ec5c2650 2024-08-01T04:19:09.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01b3680 con 0x55f8ec5cd830 2024-08-01T04:19:09.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01b28e0 con 0x55f8ec5c2f70 2024-08-01T04:19:09.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.805+0000 7f6eda46a740 10 client.0 _ll_register_callbacks cb 0x7f6ec80079c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:09.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec646f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec646f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:46160/0 (socket says 172.21.15.3:46160) 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 -- 172.21.15.3:0/977849115 learned_addr learned my addr 172.21.15.3:0/977849115 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 -- 172.21.15.3:0/977849115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 msgr2=0x55f8ec5d1a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5d1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 -- 172.21.15.3:0/977849115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 msgr2=0x55f8ec5d1500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5d1500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.806+0000 7f6ed5800640 1 -- 172.21.15.3:0/977849115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ec5d2320 con 0x55f8ec5cd830 2024-08-01T04:19:09.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.807+0000 7f6ed5800640 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec646f50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8018d80 tx=0x7f6ec800d840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5107edf67b42416 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:09.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.808+0000 7f6eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:19:09.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.808+0000 7f6eb6a00640 1 -- 172.21.15.3:0/977849115 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8038060 con 0x55f8ec5cd830 2024-08-01T04:19:09.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.808+0000 7f6eb6a00640 1 -- 172.21.15.3:0/977849115 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6ec800dd00 con 0x55f8ec5cd830 2024-08-01T04:19:09.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.809+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8ec5b9e50 con 0x55f8ec5cd830 2024-08-01T04:19:09.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.809+0000 7f6eb6a00640 1 -- 172.21.15.3:0/977849115 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6ec8026070 con 0x55f8ec5cd830 2024-08-01T04:19:09.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.810+0000 7f6eda46a740 10 client.25820 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:09.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.810+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f8ec6595c0 con 0x55f8ec5cd830 2024-08-01T04:19:09.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.810+0000 7f6eb6a00640 1 -- 172.21.15.3:0/977849115 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f8ec5b9e50 con 0x55f8ec5cd830 2024-08-01T04:19:09.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.810+0000 7f6eb6a00640 1 -- 172.21.15.3:0/977849115 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6ec8029030 con 0x55f8ec5cd830 2024-08-01T04:19:09.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.811+0000 7f6eda46a740 10 client.25820 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:09.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.811+0000 7f6eda46a740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:19:09.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.811+0000 7f6eda46a740 20 client.25820 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:09.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.811+0000 7f6eda46a740 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec659c90 0x55f8ec65c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:09.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.811+0000 7f6eda46a740 4 client.25820 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:19:09.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.811+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f8ec530540 con 0x55f8ec659c90 2024-08-01T04:19:09.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.812+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec659c90 0x55f8ec65c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:09.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.812+0000 7f6ed4e00640 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec659c90 0x55f8ec65c0f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd59e3cd7b917571 server_cookie=4dfe53723038cb8d in_seq=0 out_seq=0 2024-08-01T04:19:09.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.812+0000 7f6eb6a00640 10 client.25820 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:09.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.813+0000 7f6eb6a00640 1 -- 172.21.15.3:0/977849115 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f8ec530540 con 0x55f8ec659c90 2024-08-01T04:19:09.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.813+0000 7f6eb6a00640 10 client.25820 handle_command_reply: tid=0 2024-08-01T04:19:09.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.814+0000 7f6eda46a740 1 client.25820 shutdown 2024-08-01T04:19:09.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 msgr2=0x55f8ec646f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec646f50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8018d80 tx=0x7f6ec800d840 comp rx=0 tx=0).stop 2024-08-01T04:19:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 shutdown_connections 2024-08-01T04:19:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f8ec659c90 0x55f8ec65c0f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f8ec5cd830 0x55f8ec646f50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f8ec5c2f70 0x55f8ec5d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 --2- 172.21.15.3:0/977849115 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8ec5c2650 0x55f8ec5d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.816+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 >> 172.21.15.3:0/977849115 conn(0x55f8ec5bebe0 msgr2=0x55f8ec5caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.817+0000 7f6eb6a00640 0 client.25820 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.817+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 shutdown_connections 2024-08-01T04:19:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.817+0000 7f6eda46a740 1 -- 172.21.15.3:0/977849115 wait complete. 2024-08-01T04:19:09.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.817+0000 7f6eda46a740 20 client.25820 trim_cache size 0 max 16384 2024-08-01T04:19:09.820 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:19:09.820 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:19:09.821 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "3bae744b-f533-46e4-967c-015097f17cab", 2024-08-01T04:19:09.821 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:19:09.821 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:19:09.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6eb80a2cf0 msgr2=0x7f6eb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6eb80a2cf0 0x7f6eb80a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009cf0 tx=0x7f6ec4009830 comp rx=0 tx=0).stop 2024-08-01T04:19:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 msgr2=0x7f6ed010d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 0x7f6ed010d700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc02aa00 tx=0x7f6ebc003c10 comp rx=0 tx=0).stop 2024-08-01T04:19:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 shutdown_connections 2024-08-01T04:19:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ed0109900 0x7f6ed010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6eb80a2cf0 0x7f6eb80a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed01042a0 0x7f6ed010d700 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc02aa00 tx=0x7f6ebc003c10 comp rx=0 tx=0).stop 2024-08-01T04:19:09.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 --2- 172.21.15.3:0/862727139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ed0103900 0x7f6ed0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:09.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.819+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 >> 172.21.15.3:0/862727139 conn(0x7f6ed00ff640 msgr2=0x7f6ed00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:09.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.820+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 shutdown_connections 2024-08-01T04:19:09.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:09.821+0000 7f6ed6200640 1 -- 172.21.15.3:0/862727139 wait complete. 2024-08-01T04:19:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:10 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:19:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:10 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:10 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:19:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:10 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:10 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:19:10.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:10 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:10 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:19:10.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:10 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:10 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:19:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:10 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:10 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:19:10.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:10 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:10 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:19:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:10 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:10 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:19:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:10 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:10 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:19:10.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:10 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:19:11.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:19:11.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:19:11.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:19:11.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:11 smithi003 ceph-mon[32262]: pgmap v90: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 183 MiB/s wr, 86 op/s 2024-08-01T04:19:11.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:19:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:19:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:19:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:11 smithi163 ceph-mon[37383]: pgmap v90: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 183 MiB/s wr, 86 op/s 2024-08-01T04:19:11.859 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:19:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:19:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:19:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:19:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:19:11.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:11 smithi160 ceph-mon[37224]: pgmap v90: 97 pgs: 97 active+clean; 6.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 183 MiB/s wr, 86 op/s 2024-08-01T04:19:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 -- 172.21.15.3:0/3369048919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001076f0 msgr2=0x7ff600107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 --2- 172.21.15.3:0/3369048919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001076f0 0x7ff600107ad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009930 tx=0x7ff5f402a420 comp rx=0 tx=0).stop 2024-08-01T04:19:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 -- 172.21.15.3:0/3369048919 shutdown_connections 2024-08-01T04:19:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 --2- 172.21.15.3:0/3369048919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 --2- 172.21.15.3:0/3369048919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001016f0 0x7ff600101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 --2- 172.21.15.3:0/3369048919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001076f0 0x7ff600107ad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 -- 172.21.15.3:0/3369048919 >> 172.21.15.3:0/3369048919 conn(0x7ff6000fd470 msgr2=0x7ff6000ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 -- 172.21.15.3:0/3369048919 shutdown_connections 2024-08-01T04:19:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.517+0000 7ff607200640 1 -- 172.21.15.3:0/3369048919 wait complete. 2024-08-01T04:19:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.518+0000 7ff607200640 1 Processor -- start 2024-08-01T04:19:12.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.518+0000 7ff607200640 1 -- start start 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff607200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001016f0 0x7ff60019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff607200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff607200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001076f0 0x7ff60019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff607200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff600111af0 con 0x7ff6001016f0 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff607200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff600111970 con 0x7ff600102090 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff607200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff600111c70 con 0x7ff6001076f0 2024-08-01T04:19:12.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff606800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001076f0 0x7ff60019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff605e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001016f0 0x7ff60019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff605400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff605400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51726/0 (socket says 172.21.15.3:51726) 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff606800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001076f0 0x7ff60019eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40970/0 (socket says 172.21.15.3:40970) 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.519+0000 7ff605400640 1 -- 172.21.15.3:0/3479277539 learned_addr learned my addr 172.21.15.3:0/3479277539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff605400640 1 -- 172.21.15.3:0/3479277539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001076f0 msgr2=0x7ff60019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff605400640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001076f0 0x7ff60019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff605400640 1 -- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001016f0 msgr2=0x7ff60019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff605400640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001016f0 0x7ff60019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff605400640 1 -- 172.21.15.3:0/3479277539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60019f180 con 0x7ff600102090 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff605400640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60019a710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000e9f0 tx=0x7ff5f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff5faa00640 1 -- 172.21.15.3:0/3479277539 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000cd80 con 0x7ff600102090 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff5faa00640 1 -- 172.21.15.3:0/3479277539 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0010910 con 0x7ff600102090 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff5faa00640 1 -- 172.21.15.3:0/3479277539 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0018950 con 0x7ff600102090 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60019f470 con 0x7ff600102090 2024-08-01T04:19:12.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.520+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6001a6d50 con 0x7ff600102090 2024-08-01T04:19:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.523+0000 7ff60b24b740 1 Processor -- start 2024-08-01T04:19:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 -- start start 2024-08-01T04:19:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb8750f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff5faa00640 1 -- 172.21.15.3:0/3479277539 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff5f0010ac0 con 0x7ff600102090 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750f6a0 0x55fb87519a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb87519f60 0x55fb8751c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fb87474bf0 con 0x55fb8750ed80 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fb8740cbd0 con 0x55fb8750f6a0 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff60b24b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fb87470bb0 con 0x55fb87519f60 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff606800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb8750f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff606800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb8750f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40276/0 (socket says 172.21.15.3:40276) 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff605e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750f6a0 0x55fb87519a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.524+0000 7ff606800640 1 -- 172.21.15.3:0/1509392798 learned_addr learned my addr 172.21.15.3:0/1509392798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:12.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605400640 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb87519f60 0x55fb8751c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605e00640 1 -- 172.21.15.3:0/1509392798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb87519f60 msgr2=0x55fb8751c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605e00640 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb87519f60 0x55fb8751c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605e00640 1 -- 172.21.15.3:0/1509392798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 msgr2=0x55fb8750f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605e00640 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb8750f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605e00640 1 -- 172.21.15.3:0/1509392798 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb874138a0 con 0x55fb8750f6a0 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff605e00640 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750f6a0 0x55fb87519a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4005940 tx=0x7ff5f4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f0780f2cf30fa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.525+0000 7ff5ef000640 1 -- 172.21.15.3:0/1509392798 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4040070 con 0x55fb8750f6a0 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff5ef000640 1 -- 172.21.15.3:0/1509392798 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4031870 con 0x55fb8750f6a0 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff5ef000640 1 -- 172.21.15.3:0/1509392798 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4035040 con 0x55fb8750f6a0 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff5faa00640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff5e80a2cf0 0x7ff5e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff5faa00640 1 -- 172.21.15.3:0/3479277539 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff5f00cf830 con 0x7ff600102090 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff606800640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff5e80a2cf0 0x7ff5e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff60b24b740 1 -- 172.21.15.3:0/1509392798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750f6a0 msgr2=0x55fb87519a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.526+0000 7ff60b24b740 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750f6a0 0x55fb87519a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4005940 tx=0x7ff5f4005bd0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 -- 172.21.15.3:0/1509392798 shutdown_connections 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb87519f60 0x55fb8751c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750f6a0 0x55fb87519a20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 --2- 172.21.15.3:0/1509392798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb8750f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 -- 172.21.15.3:0/1509392798 >> 172.21.15.3:0/1509392798 conn(0x55fb8750b310 msgr2=0x55fb8750b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 -- 172.21.15.3:0/1509392798 shutdown_connections 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 -- 172.21.15.3:0/1509392798 wait complete. 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff606800640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff5e80a2cf0 0x7ff5e80a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc011630 tx=0x7ff5fc009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.527+0000 7ff60b24b740 1 -- start start 2024-08-01T04:19:12.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 Processor -- start 2024-08-01T04:19:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb87516e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87517370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb875178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fb8751cb30 con 0x55fb8750ed80 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fb8742fd40 con 0x55fb87519f60 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fb8751ccb0 con 0x55fb8750f6a0 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.528+0000 7ff60b24b740 10 client.0 _ll_register_callbacks cb 0x55fb873f8b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff606800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb875178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87517370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb87516e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff606800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb875178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51754/0 (socket says 172.21.15.3:51754) 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87517370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40996/0 (socket says 172.21.15.3:40996) 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff606800640 1 -- 172.21.15.3:0/2350957524 learned_addr learned my addr 172.21.15.3:0/2350957524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 -- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 msgr2=0x55fb875178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb875178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 -- 172.21.15.3:0/2350957524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 msgr2=0x55fb87516e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb87516e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 -- 172.21.15.3:0/2350957524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb871e63c0 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605e00640 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb87516e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff605400640 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87517370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00ccee0 tx=0x7ff5f009b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586838bbe329fba0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.529+0000 7ff5e3400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff5e3400640 1 -- 172.21.15.3:0/2350957524 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00cd360 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f4031870 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff5e3400640 1 -- 172.21.15.3:0/2350957524 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff5f009b1b0 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff5e3400640 1 -- 172.21.15.3:0/2350957524 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5f009b3b0 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 10 client.25813 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb87531020 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff5e3400640 1 -- 172.21.15.3:0/2350957524 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4031870 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff5e3400640 1 -- 172.21.15.3:0/2350957524 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff5f0097ab0 con 0x55fb8750f6a0 2024-08-01T04:19:12.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 10 client.25813 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 20 client.25813 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.530+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb87532430 0x55fb875348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.531+0000 7ff60b24b740 4 client.25813 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.531+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb87482160 con 0x55fb87532430 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.531+0000 7ff605e00640 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb87532430 0x55fb875348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.532+0000 7ff605e00640 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb87532430 0x55fb875348a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23aa45b770e573c7 server_cookie=5f8156d5cad4046c in_seq=0 out_seq=0 2024-08-01T04:19:12.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.532+0000 7ff5e3400640 10 client.25813 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.532+0000 7ff5e3400640 1 -- 172.21.15.3:0/2350957524 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb87482160 con 0x55fb87532430 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.532+0000 7ff5e3400640 10 client.25813 handle_command_reply: tid=0 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.532+0000 7ff60b24b740 1 client.25813 shutdown 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 msgr2=0x55fb87517370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87517370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00ccee0 tx=0x7ff5f009b780 comp rx=0 tx=0).stop 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 shutdown_connections 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb87532430 0x55fb875348a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb875178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87517370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2350957524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750ed80 0x55fb87516e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 >> 172.21.15.3:0/2350957524 conn(0x55fb8750b310 msgr2=0x55fb87511040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff5e3400640 0 client.25813 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 shutdown_connections 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 1 -- 172.21.15.3:0/2350957524 wait complete. 2024-08-01T04:19:12.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.533+0000 7ff60b24b740 20 client.25813 trim_cache size 0 max 16384 2024-08-01T04:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.552+0000 7ff60b24b740 1 Processor -- start 2024-08-01T04:19:12.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.552+0000 7ff60b24b740 1 -- start start 2024-08-01T04:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.552+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750ed80 0x55fb8753a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.552+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750f6a0 0x55fb8753a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb87514090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff60b24b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001adce0 con 0x55fb8750f6a0 2024-08-01T04:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff60b24b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001ab5b0 con 0x55fb87519f60 2024-08-01T04:19:12.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff60b24b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001a9420 con 0x55fb8750ed80 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb87514090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff605400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750f6a0 0x55fb8753a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb87514090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51762/0 (socket says 172.21.15.3:51762) 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 -- 172.21.15.3:0/2361416638 learned_addr learned my addr 172.21.15.3:0/2361416638 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff605e00640 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750ed80 0x55fb8753a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 -- 172.21.15.3:0/2361416638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750ed80 msgr2=0x55fb8753a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750ed80 0x55fb8753a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 -- 172.21.15.3:0/2361416638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750f6a0 msgr2=0x55fb8753a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750f6a0 0x55fb8753a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.553+0000 7ff606800640 1 -- 172.21.15.3:0/2361416638 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb8750f4d0 con 0x55fb87519f60 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.554+0000 7ff606800640 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb87514090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc003e60 tx=0x7ff5fc01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47cd4c1a4f4a9dc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.554+0000 7ff5ee600640 1 -- 172.21.15.3:0/2361416638 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00f620 con 0x55fb87519f60 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.554+0000 7ff5ee600640 1 -- 172.21.15.3:0/2361416638 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc00fca0 con 0x55fb87519f60 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.554+0000 7ff5ee600640 1 -- 172.21.15.3:0/2361416638 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc029e30 con 0x55fb87519f60 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 -- 172.21.15.3:0/2361416638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 msgr2=0x55fb87514090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb87514090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc003e60 tx=0x7ff5fc01a000 comp rx=0 tx=0).stop 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 -- 172.21.15.3:0/2361416638 shutdown_connections 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb87519f60 0x55fb87514090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb8750f6a0 0x55fb8753a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 --2- 172.21.15.3:0/2361416638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750ed80 0x55fb8753a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 -- 172.21.15.3:0/2361416638 >> 172.21.15.3:0/2361416638 conn(0x55fb8750b310 msgr2=0x55fb8750a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 -- 172.21.15.3:0/2361416638 shutdown_connections 2024-08-01T04:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.555+0000 7ff60b24b740 1 -- 172.21.15.3:0/2361416638 wait complete. 2024-08-01T04:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.556+0000 7ff60b24b740 1 -- start start 2024-08-01T04:19:12.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.556+0000 7ff60b24b740 1 Processor -- start 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750ed80 0x55fb875265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87526b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb87519f60 0x55fb87527070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001a9a70 con 0x55fb87519f60 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001aaee0 con 0x55fb8750ed80 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001aa140 con 0x55fb8750f6a0 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff60b24b740 10 client.0 _ll_register_callbacks cb 0x7ff5f000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750ed80 0x55fb875265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750ed80 0x55fb875265f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51766/0 (socket says 172.21.15.3:51766) 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605e00640 1 -- 172.21.15.3:0/30553619 learned_addr learned my addr 172.21.15.3:0/30553619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605400640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87526b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff606800640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb87519f60 0x55fb87527070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605400640 1 -- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750ed80 msgr2=0x55fb875265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605400640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750ed80 0x55fb875265f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605400640 1 -- 172.21.15.3:0/30553619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb87519f60 msgr2=0x55fb87527070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605400640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb87519f60 0x55fb87527070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.557+0000 7ff605400640 1 -- 172.21.15.3:0/30553619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb8759c1c0 con 0x55fb8750f6a0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff605400640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87526b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0000c00 tx=0x7ff5f00cd410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff78fb66c1a238f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff5e3400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff5e3400640 1 -- 172.21.15.3:0/30553619 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f009bb30 con 0x55fb8750f6a0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff5e3400640 1 -- 172.21.15.3:0/30553619 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7ff5f009b1b0 con 0x55fb8750f6a0 2024-08-01T04:19:12.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5fc00fca0 con 0x55fb8750f6a0 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff5e3400640 1 -- 172.21.15.3:0/30553619 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5f0018d00 con 0x55fb8750f6a0 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff60b24b740 10 client.25819 fetch_fsmap learned FSMap version 27 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb8759d4c0 con 0x55fb8750f6a0 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff5e3400640 1 -- 172.21.15.3:0/30553619 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc00fca0 con 0x55fb8750f6a0 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.558+0000 7ff5e3400640 1 -- 172.21.15.3:0/30553619 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff5f00ba370 con 0x55fb8750f6a0 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff60b24b740 10 client.25819 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff60b24b740 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff60b24b740 20 client.25819 populate_metadata read hostname 'smithi003' 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff60b24b740 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb8759dca0 0x55fb875a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff60b24b740 4 client.25819 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb8747c670 con 0x55fb8759dca0 2024-08-01T04:19:12.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.559+0000 7ff605e00640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb8759dca0 0x55fb875a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:12.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff605e00640 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb8759dca0 0x55fb875a00c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b80013d73a33223 server_cookie=b78dca171d68f91e in_seq=0 out_seq=0 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff5e3400640 10 client.25819 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff5e3400640 1 -- 172.21.15.3:0/30553619 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fb8747c670 con 0x55fb8759dca0 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff5e3400640 10 client.25819 handle_command_reply: tid=0 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 client.25819 shutdown 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 msgr2=0x55fb87526b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87526b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0000c00 tx=0x7ff5f00cd410 comp rx=0 tx=0).stop 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 shutdown_connections 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fb8759dca0 0x55fb875a00c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb87519f60 0x55fb87527070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb8750f6a0 0x55fb87526b30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 --2- 172.21.15.3:0/30553619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb8750ed80 0x55fb875265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 >> 172.21.15.3:0/30553619 conn(0x55fb8750b310 msgr2=0x55fb87510820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:12.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff5e3400640 0 client.25819 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 shutdown_connections 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 1 -- 172.21.15.3:0/30553619 wait complete. 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.560+0000 7ff60b24b740 20 client.25819 trim_cache size 0 max 16384 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout: "3bae744b-f533-46e4-967c-015097f17cab": { 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "3bae744b-f533-46e4-967c-015097f17cab", 2024-08-01T04:19:12.566 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:19:12.567 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:19:12.567 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:19:12.567 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:19:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff5e80a2cf0 msgr2=0x7ff5e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff5e80a2cf0 0x7ff5e80a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc011630 tx=0x7ff5fc009500 comp rx=0 tx=0).stop 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 msgr2=0x7ff60019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60019a710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000e9f0 tx=0x7ff5f000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 shutdown_connections 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7ff5e80a2cf0 0x7ff5e80a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6001076f0 0x7ff60019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff600102090 0x7ff60019a710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 --2- 172.21.15.3:0/3479277539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff6001016f0 0x7ff60019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 >> 172.21.15.3:0/3479277539 conn(0x7ff6000fd470 msgr2=0x7ff60010d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.561+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 shutdown_connections 2024-08-01T04:19:12.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:12.562+0000 7ff607200640 1 -- 172.21.15.3:0/3479277539 wait complete. 2024-08-01T04:19:13.655 INFO:tasks.cephfs.filesystem:scrub status for tag:3bae744b-f533-46e4-967c-015097f17cab - {'path': '/', 'tag': '3bae744b-f533-46e4-967c-015097f17cab', 'options': 'recursive,force'} 2024-08-01T04:19:13.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:13 smithi163 ceph-mon[37383]: pgmap v91: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 265 MiB/s wr, 116 op/s 2024-08-01T04:19:13.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:13 smithi003 ceph-mon[32262]: pgmap v91: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 265 MiB/s wr, 116 op/s 2024-08-01T04:19:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:13 smithi160 ceph-mon[37224]: pgmap v91: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 265 MiB/s wr, 116 op/s 2024-08-01T04:19:15.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:15 smithi163 ceph-mon[37383]: pgmap v92: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 212 MiB/s wr, 76 op/s 2024-08-01T04:19:15.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:15 smithi003 ceph-mon[32262]: pgmap v92: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 212 MiB/s wr, 76 op/s 2024-08-01T04:19:15.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:15 smithi160 ceph-mon[37224]: pgmap v92: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 212 MiB/s wr, 76 op/s 2024-08-01T04:19:17.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:17 smithi163 ceph-mon[37383]: pgmap v93: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 191 MiB/s wr, 71 op/s 2024-08-01T04:19:17.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:17 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:19:17.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:17 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:19:17.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:17 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:19:17.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:17 smithi003 ceph-mon[32262]: pgmap v93: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 191 MiB/s wr, 71 op/s 2024-08-01T04:19:17.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:17 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:19:17.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:17 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:19:17.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:17 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:19:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:17 smithi160 ceph-mon[37224]: pgmap v93: 97 pgs: 97 active+clean; 7.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 191 MiB/s wr, 71 op/s 2024-08-01T04:19:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:17 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:19:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:17 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:19:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:17 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:19:19.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:19 smithi163 ceph-mon[37383]: pgmap v94: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 289 MiB/s wr, 109 op/s 2024-08-01T04:19:19.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:19 smithi003 ceph-mon[32262]: pgmap v94: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 289 MiB/s wr, 109 op/s 2024-08-01T04:19:19.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:19 smithi160 ceph-mon[37224]: pgmap v94: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 289 MiB/s wr, 109 op/s 2024-08-01T04:19:21.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:21 smithi163 ceph-mon[37383]: pgmap v95: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 203 MiB/s wr, 75 op/s 2024-08-01T04:19:21.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:21 smithi003 ceph-mon[32262]: pgmap v95: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 203 MiB/s wr, 75 op/s 2024-08-01T04:19:21.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:21 smithi160 ceph-mon[37224]: pgmap v95: 97 pgs: 97 active+clean; 8.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 203 MiB/s wr, 75 op/s 2024-08-01T04:19:22.201 INFO:tasks.workunit.client.0.smithi003.stdout:fs setup took 49 secs 2024-08-01T04:19:23.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:23 smithi003 ceph-mon[32262]: pgmap v96: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 297 MiB/s wr, 111 op/s 2024-08-01T04:19:23.798 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:23 smithi163 ceph-mon[37383]: pgmap v96: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 297 MiB/s wr, 111 op/s 2024-08-01T04:19:23.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:23 smithi160 ceph-mon[37224]: pgmap v96: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 297 MiB/s wr, 111 op/s 2024-08-01T04:19:25.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:25 smithi163 ceph-mon[37383]: pgmap v97: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 226 MiB/s wr, 83 op/s 2024-08-01T04:19:25.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:25 smithi003 ceph-mon[32262]: pgmap v97: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 226 MiB/s wr, 83 op/s 2024-08-01T04:19:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:25 smithi160 ceph-mon[37224]: pgmap v97: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 226 MiB/s wr, 83 op/s 2024-08-01T04:19:27.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:27 smithi163 ceph-mon[37383]: pgmap v98: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 203 MiB/s wr, 77 op/s 2024-08-01T04:19:27.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:27 smithi003 ceph-mon[32262]: pgmap v98: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 203 MiB/s wr, 77 op/s 2024-08-01T04:19:27.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:27 smithi160 ceph-mon[37224]: pgmap v98: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 203 MiB/s wr, 77 op/s 2024-08-01T04:19:29.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:29 smithi163 ceph-mon[37383]: pgmap v99: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 318 MiB/s wr, 110 op/s 2024-08-01T04:19:29.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:29 smithi003 ceph-mon[32262]: pgmap v99: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 318 MiB/s wr, 110 op/s 2024-08-01T04:19:29.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:29 smithi160 ceph-mon[37224]: pgmap v99: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 318 MiB/s wr, 110 op/s 2024-08-01T04:19:31.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:31 smithi163 ceph-mon[37383]: pgmap v100: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 220 MiB/s wr, 71 op/s 2024-08-01T04:19:31.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:31 smithi003 ceph-mon[32262]: pgmap v100: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 220 MiB/s wr, 71 op/s 2024-08-01T04:19:31.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:31 smithi160 ceph-mon[37224]: pgmap v100: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 220 MiB/s wr, 71 op/s 2024-08-01T04:19:32.265 INFO:tasks.workunit.client.0.smithi003.stdout:Syncing()...10 sec 2024-08-01T04:19:32.267 INFO:tasks.workunit.client.0.smithi003.stdout:Starting Actual Benchmark At: Thu Aug 1 04:19:32 2024 2024-08-01T04:19:32.267 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-08-01T04:19:33.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:33 smithi163 ceph-mon[37383]: pgmap v101: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 329 MiB/s wr, 102 op/s 2024-08-01T04:19:33.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:33 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:19:33.822 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:33 smithi003 ceph-mon[32262]: pgmap v101: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 329 MiB/s wr, 102 op/s 2024-08-01T04:19:33.823 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:33 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:19:33.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:33 smithi160 ceph-mon[37224]: pgmap v101: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 329 MiB/s wr, 102 op/s 2024-08-01T04:19:33.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:33 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:19:35.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:35 smithi163 ceph-mon[37383]: pgmap v102: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 252 MiB/s wr, 70 op/s 2024-08-01T04:19:35.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:35 smithi003 ceph-mon[32262]: pgmap v102: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 252 MiB/s wr, 70 op/s 2024-08-01T04:19:35.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:35 smithi160 ceph-mon[37224]: pgmap v102: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 252 MiB/s wr, 70 op/s 2024-08-01T04:19:37.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:37 smithi163 ceph-mon[37383]: pgmap v103: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 241 MiB/s wr, 68 op/s 2024-08-01T04:19:37.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:19:37.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:19:37.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:37 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:19:37.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:37 smithi003 ceph-mon[32262]: pgmap v103: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 241 MiB/s wr, 68 op/s 2024-08-01T04:19:37.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:19:37.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:19:37.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:37 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:19:37.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:37 smithi160 ceph-mon[37224]: pgmap v103: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 241 MiB/s wr, 68 op/s 2024-08-01T04:19:37.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:19:37.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:19:37.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:37 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:19:39.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:39 smithi163 ceph-mon[37383]: pgmap v104: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 278 MiB/s wr, 4.89k op/s 2024-08-01T04:19:39.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:39 smithi003 ceph-mon[32262]: pgmap v104: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 278 MiB/s wr, 4.89k op/s 2024-08-01T04:19:39.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:39 smithi160 ceph-mon[37224]: pgmap v104: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 278 MiB/s wr, 4.89k op/s 2024-08-01T04:19:41.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:41 smithi163 ceph-mon[37383]: pgmap v105: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 163 MiB/s wr, 4.86k op/s 2024-08-01T04:19:41.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:41 smithi003 ceph-mon[32262]: pgmap v105: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 163 MiB/s wr, 4.86k op/s 2024-08-01T04:19:41.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:41 smithi160 ceph-mon[37224]: pgmap v105: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 163 MiB/s wr, 4.86k op/s 2024-08-01T04:19:42.436 INFO:tasks.quiescer.fs.[cephfs]:Total duration (47.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-01T04:19:42.437 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.55 2024-08-01T04:19:42.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:19:43.625 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:19:43.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:19:43.852 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:43 smithi003 ceph-mon[32262]: pgmap v106: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 199 MiB/s wr, 10.09k op/s 2024-08-01T04:19:43.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:43 smithi160 ceph-mon[37224]: pgmap v106: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 199 MiB/s wr, 10.09k op/s 2024-08-01T04:19:44.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:43 smithi163 ceph-mon[37383]: pgmap v106: 97 pgs: 97 active+clean; 13 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 199 MiB/s wr, 10.09k op/s 2024-08-01T04:19:45.847 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:45 smithi163 ceph-mon[37383]: pgmap v107: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 105 MiB/s wr, 12.32k op/s 2024-08-01T04:19:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:45 smithi160 ceph-mon[37224]: pgmap v107: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 105 MiB/s wr, 12.32k op/s 2024-08-01T04:19:46.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:45 smithi003 ceph-mon[32262]: pgmap v107: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 105 MiB/s wr, 12.32k op/s 2024-08-01T04:19:47.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:47 smithi160 ceph-mon[37224]: pgmap v108: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 96 MiB/s wr, 13.49k op/s 2024-08-01T04:19:48.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:47 smithi163 ceph-mon[37383]: pgmap v108: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 96 MiB/s wr, 13.49k op/s 2024-08-01T04:19:48.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:47 smithi003 ceph-mon[32262]: pgmap v108: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 96 MiB/s wr, 13.49k op/s 2024-08-01T04:19:49.801 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:49 smithi003 ceph-mon[32262]: pgmap v109: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 138 MiB/s wr, 19.55k op/s 2024-08-01T04:19:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:49 smithi160 ceph-mon[37224]: pgmap v109: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 138 MiB/s wr, 19.55k op/s 2024-08-01T04:19:50.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:49 smithi163 ceph-mon[37383]: pgmap v109: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 138 MiB/s wr, 19.55k op/s 2024-08-01T04:19:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:51 smithi160 ceph-mon[37224]: pgmap v110: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 115 MiB/s wr, 16.66k op/s 2024-08-01T04:19:52.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:51 smithi163 ceph-mon[37383]: pgmap v110: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 115 MiB/s wr, 16.66k op/s 2024-08-01T04:19:52.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:51 smithi003 ceph-mon[32262]: pgmap v110: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 115 MiB/s wr, 16.66k op/s 2024-08-01T04:19:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:53 smithi160 ceph-mon[37224]: pgmap v111: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 147 MiB/s wr, 21.00k op/s 2024-08-01T04:19:53.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:53 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3280559549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:19:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:53 smithi163 ceph-mon[37383]: pgmap v111: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 147 MiB/s wr, 21.00k op/s 2024-08-01T04:19:54.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:53 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3280559549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:19:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:53 smithi003 ceph-mon[32262]: pgmap v111: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 147 MiB/s wr, 21.00k op/s 2024-08-01T04:19:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:53 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3280559549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:19:55.609 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:19:55.610 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-08-01T04:18:17:181881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:18:16.489018+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:19:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 -- 172.21.15.3:0/981774932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84100790 msgr2=0x7f5b84100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 --2- 172.21.15.3:0/981774932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84100790 0x7f5b84100bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009930 tx=0x7f5b7002a420 comp rx=0 tx=0).stop 2024-08-01T04:19:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 -- 172.21.15.3:0/981774932 shutdown_connections 2024-08-01T04:19:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 --2- 172.21.15.3:0/981774932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b8410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 --2- 172.21.15.3:0/981774932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84100790 0x7f5b84100bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 --2- 172.21.15.3:0/981774932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b84106790 0x7f5b84106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 -- 172.21.15.3:0/981774932 >> 172.21.15.3:0/981774932 conn(0x7f5b840fc4b0 msgr2=0x7f5b840fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 -- 172.21.15.3:0/981774932 shutdown_connections 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.710+0000 7f5b88e00640 1 -- 172.21.15.3:0/981774932 wait complete. 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 Processor -- start 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 -- start start 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b84100790 0x7f5b841a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b841a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84106790 0x7f5b8419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84110be0 con 0x7f5b84106790 2024-08-01T04:19:55.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84110a60 con 0x7f5b84100790 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.712+0000 7f5b88e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84110d60 con 0x7f5b84101130 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b841a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b841a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:37630/0 (socket says 172.21.15.3:37630) 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 -- 172.21.15.3:0/3280559549 learned_addr learned my addr 172.21.15.3:0/3280559549 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b84100790 0x7f5b841a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b82800640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84106790 0x7f5b8419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 -- 172.21.15.3:0/3280559549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b84100790 msgr2=0x7f5b841a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b84100790 0x7f5b841a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 -- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84106790 msgr2=0x7f5b8419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84106790 0x7f5b8419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b81400640 1 -- 172.21.15.3:0/3280559549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8419ef90 con 0x7f5b84101130 2024-08-01T04:19:55.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.713+0000 7f5b82800640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84106790 0x7f5b8419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.714+0000 7f5b81400640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b841a3920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b7002a940 tx=0x7f5b70003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.714+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7003f070 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.714+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8419f220 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.714+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b70031c90 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.714+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70035030 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.715+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8406b940 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.716+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b84101ed0 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.718+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5b70042070 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.720+0000 7f5b7aa00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b680a2cf0 0x7f5b680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.720+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f5b700efc10 con 0x7f5b84101130 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.720+0000 7f5b81e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b680a2cf0 0x7f5b680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.721+0000 7f5b81e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b680a2cf0 0x7f5b680a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b8419fe80 tx=0x7f5b74005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:19:55.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:52.726+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5b700bc790 con 0x7f5b84101130 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.165+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b84100bf0 con 0x7f5b84101130 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.166+0000 7f5b7aa00640 1 -- 172.21.15.3:0/3280559549 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11911 (secure 0 0 0) 0x7f5b700bc090 con 0x7f5b84101130 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b680a2cf0 msgr2=0x7f5b680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b680a2cf0 0x7f5b680a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5b8419fe80 tx=0x7f5b74005f90 comp rx=0 tx=0).stop 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 msgr2=0x7f5b841a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b841a3920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b7002a940 tx=0x7f5b70003b90 comp rx=0 tx=0).stop 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 shutdown_connections 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b84106790 0x7f5b8419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b680a2cf0 0x7f5b680a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b84101130 0x7f5b841a3920 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 --2- 172.21.15.3:0/3280559549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b84100790 0x7f5b841a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:19:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.171+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 >> 172.21.15.3:0/3280559549 conn(0x7f5b840fc4b0 msgr2=0x7f5b8406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:19:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.175+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 shutdown_connections 2024-08-01T04:19:55.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:19:53.175+0000 7f5b88e00640 1 -- 172.21.15.3:0/3280559549 wait complete. 2024-08-01T04:19:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:55 smithi160 ceph-mon[37224]: pgmap v112: 97 pgs: 97 active+clean; 14 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 118 MiB/s wr, 16.72k op/s 2024-08-01T04:19:56.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:55 smithi163 ceph-mon[37383]: pgmap v112: 97 pgs: 97 active+clean; 14 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 118 MiB/s wr, 16.72k op/s 2024-08-01T04:19:56.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:55 smithi003 ceph-mon[32262]: pgmap v112: 97 pgs: 97 active+clean; 14 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 118 MiB/s wr, 16.72k op/s 2024-08-01T04:19:58.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:57 smithi163 ceph-mon[37383]: pgmap v113: 97 pgs: 97 active+clean; 14 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 118 MiB/s wr, 16.47k op/s 2024-08-01T04:19:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:57 smithi003 ceph-mon[32262]: pgmap v113: 97 pgs: 97 active+clean; 14 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 118 MiB/s wr, 16.47k op/s 2024-08-01T04:19:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:57 smithi160 ceph-mon[37224]: pgmap v113: 97 pgs: 97 active+clean; 14 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 118 MiB/s wr, 16.47k op/s 2024-08-01T04:19:59.802 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:19:59 smithi003 ceph-mon[32262]: pgmap v114: 97 pgs: 97 active+clean; 15 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 177 MiB/s wr, 24.51k op/s 2024-08-01T04:20:00.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:19:59 smithi163 ceph-mon[37383]: pgmap v114: 97 pgs: 97 active+clean; 15 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 177 MiB/s wr, 24.51k op/s 2024-08-01T04:20:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:19:59 smithi160 ceph-mon[37224]: pgmap v114: 97 pgs: 97 active+clean; 15 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 177 MiB/s wr, 24.51k op/s 2024-08-01T04:20:01.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:00 smithi163 ceph-mon[37383]: overall HEALTH_OK 2024-08-01T04:20:01.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:00 smithi003 ceph-mon[32262]: overall HEALTH_OK 2024-08-01T04:20:01.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:00 smithi160 ceph-mon[37224]: overall HEALTH_OK 2024-08-01T04:20:01.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:01 smithi003 ceph-mon[32262]: pgmap v115: 97 pgs: 97 active+clean; 15 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 141 MiB/s wr, 19.21k op/s 2024-08-01T04:20:02.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:01 smithi163 ceph-mon[37383]: pgmap v115: 97 pgs: 97 active+clean; 15 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 141 MiB/s wr, 19.21k op/s 2024-08-01T04:20:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:01 smithi160 ceph-mon[37224]: pgmap v115: 97 pgs: 97 active+clean; 15 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 141 MiB/s wr, 19.21k op/s 2024-08-01T04:20:03.020 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:03.075 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db / --timeout 90 --expiration 143.55 --await-for 45.0' 2024-08-01T04:20:03.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db / --timeout 90 --expiration 143.55 --await-for 45.0 2024-08-01T04:20:03.961 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:03 smithi003 ceph-mon[32262]: pgmap v116: 97 pgs: 97 active+clean; 15 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 144 MiB/s wr, 19.61k op/s 2024-08-01T04:20:04.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:03 smithi163 ceph-mon[37383]: pgmap v116: 97 pgs: 97 active+clean; 15 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 144 MiB/s wr, 19.61k op/s 2024-08-01T04:20:04.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:03 smithi160 ceph-mon[37224]: pgmap v116: 97 pgs: 97 active+clean; 15 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 144 MiB/s wr, 19.61k op/s 2024-08-01T04:20:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.385+0000 7f2bb7600640 1 -- 172.21.15.3:0/1167000424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0112760 msgr2=0x7f2bb0114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.385+0000 7f2bb7600640 1 --2- 172.21.15.3:0/1167000424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0112760 0x7f2bb0114b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009ca0 tx=0x7f2ba802a8f0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.386+0000 7f2bb7600640 1 -- 172.21.15.3:0/1167000424 shutdown_connections 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.386+0000 7f2bb7600640 1 --2- 172.21.15.3:0/1167000424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0112760 0x7f2bb0114b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.386+0000 7f2bb7600640 1 --2- 172.21.15.3:0/1167000424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.386+0000 7f2bb7600640 1 --2- 172.21.15.3:0/1167000424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0076450 0x7f2bb0076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.386+0000 7f2bb7600640 1 -- 172.21.15.3:0/1167000424 >> 172.21.15.3:0/1167000424 conn(0x7f2bb00ff7c0 msgr2=0x7f2bb0101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.388+0000 7f2bb7600640 1 -- 172.21.15.3:0/1167000424 shutdown_connections 2024-08-01T04:20:04.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.388+0000 7f2bb7600640 1 -- 172.21.15.3:0/1167000424 wait complete. 2024-08-01T04:20:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.389+0000 7f2bb7600640 1 Processor -- start 2024-08-01T04:20:04.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.390+0000 7f2bb7600640 1 -- start start 2024-08-01T04:20:04.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.391+0000 7f2bb7600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 0x7f2bb01085c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.393+0000 7f2bb7600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0108b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.393+0000 7f2bb7600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0112760 0x7f2bb01039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.393+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 0x7f2bb01085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 0x7f2bb01085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41030/0 (socket says 172.21.15.3:41030) 2024-08-01T04:20:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2bb7600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb006c390 con 0x7f2bb0076d70 2024-08-01T04:20:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2bb7600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb006c210 con 0x7f2bb0112760 2024-08-01T04:20:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2bb7600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb006c510 con 0x7f2bb0076450 2024-08-01T04:20:04.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46104/0 (socket says 172.21.15.3:46104) 2024-08-01T04:20:04.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2bb4c00640 1 -- 172.21.15.3:0/3209562114 learned_addr learned my addr 172.21.15.3:0/3209562114 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:04.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.394+0000 7f2bb5600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0112760 0x7f2bb01039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.397+0000 7f2bb4c00640 1 -- 172.21.15.3:0/3209562114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0112760 msgr2=0x7f2bb01039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.397+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0112760 0x7f2bb01039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.397+0000 7f2bb4c00640 1 -- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 msgr2=0x7f2bb0108b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.400+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0108b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.400+0000 7f2bb4c00640 1 -- 172.21.15.3:0/3209562114 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb01042b0 con 0x7f2bb0076450 2024-08-01T04:20:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.401+0000 7f2bb5600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0112760 0x7f2bb01039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.401+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:04.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.402+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 0x7f2bb01085c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b450 tx=0x7f2ba000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.402+0000 7f2bae000640 1 -- 172.21.15.3:0/3209562114 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0015020 con 0x7f2bb0076450 2024-08-01T04:20:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.402+0000 7f2bae000640 1 -- 172.21.15.3:0/3209562114 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba0011d70 con 0x7f2bb0076450 2024-08-01T04:20:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.402+0000 7f2bae000640 1 -- 172.21.15.3:0/3209562114 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000f340 con 0x7f2bb0076450 2024-08-01T04:20:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.402+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb0104540 con 0x7f2bb0076450 2024-08-01T04:20:04.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.403+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb0110280 con 0x7f2bb0076450 2024-08-01T04:20:04.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.411+0000 7f2bae000640 1 -- 172.21.15.3:0/3209562114 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2ba0018020 con 0x7f2bb0076450 2024-08-01T04:20:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.414+0000 7f2bae000640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2b940a2f00 0x7f2b940a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.414+0000 7f2bae000640 1 -- 172.21.15.3:0/3209562114 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f2ba00cee90 con 0x7f2bb0076450 2024-08-01T04:20:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.431+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2b940a2f00 0x7f2b940a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 Processor -- start 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 -- start start 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc3886f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc3891830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc3894160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557fc37ecbf0 con 0x557fc3886b90 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557fc3784bd0 con 0x557fc38874b0 2024-08-01T04:20:04.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.432+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557fc37e8bb0 con 0x557fc3891d70 2024-08-01T04:20:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.433+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc3886f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.433+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc3891830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.433+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc3891830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51062/0 (socket says 172.21.15.3:51062) 2024-08-01T04:20:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.433+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2502026667 learned_addr learned my addr 172.21.15.3:0/2502026667 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:04.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.433+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc3894160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.433+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2b940a2f00 0x7f2b940a53c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0104ec0 tx=0x7f2b98005ec0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.434+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2502026667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 msgr2=0x557fc3894160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.434+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc3894160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.434+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2502026667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 msgr2=0x557fc3886f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.434+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc3886f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.434+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2502026667 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc378b8a0 con 0x557fc38874b0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.435+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc3891830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800ee40 tx=0x7f2b98011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783062a480787494 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.435+0000 7f2ba5c00640 1 -- 172.21.15.3:0/2502026667 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98021070 con 0x557fc38874b0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.436+0000 7f2ba5c00640 1 -- 172.21.15.3:0/2502026667 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b9800ee70 con 0x557fc38874b0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.436+0000 7f2ba5c00640 1 -- 172.21.15.3:0/2502026667 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98021070 con 0x557fc38874b0 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.437+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2502026667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 msgr2=0x557fc3891830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.438+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc3891830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800ee40 tx=0x7f2b98011000 comp rx=0 tx=0).stop 2024-08-01T04:20:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2502026667 shutdown_connections 2024-08-01T04:20:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc3894160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc3891830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2502026667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc3886f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2502026667 >> 172.21.15.3:0/2502026667 conn(0x557fc3883120 msgr2=0x557fc3883500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2502026667 shutdown_connections 2024-08-01T04:20:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.439+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2502026667 wait complete. 2024-08-01T04:20:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.440+0000 7f2bb9ffb740 1 -- start start 2024-08-01T04:20:04.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.440+0000 7f2bb9ffb740 1 Processor -- start 2024-08-01T04:20:04.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.442+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc388eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.443+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc388f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.443+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc388f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.443+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557fc3894940 con 0x557fc3886b90 2024-08-01T04:20:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.443+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557fc37a7d40 con 0x557fc38874b0 2024-08-01T04:20:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.443+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557fc3894ac0 con 0x557fc3891d70 2024-08-01T04:20:04.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.443+0000 7f2bb9ffb740 10 client.0 _ll_register_callbacks cb 0x557fc3770b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.448+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc388eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.448+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc388eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46122/0 (socket says 172.21.15.3:46122) 2024-08-01T04:20:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.448+0000 7f2bb4c00640 1 -- 172.21.15.3:0/2750122545 learned_addr learned my addr 172.21.15.3:0/2750122545 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.448+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc388f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.448+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc388f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2bb5600640 1 -- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 msgr2=0x557fc388f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc388f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2bb5600640 1 -- 172.21.15.3:0/2750122545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 msgr2=0x557fc388eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc388eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2bb5600640 1 -- 172.21.15.3:0/2750122545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc355e3c0 con 0x557fc3891d70 2024-08-01T04:20:04.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc388f620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0079330 tx=0x7f2ba8003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2144361b85a18b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.449+0000 7f2b9e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.450+0000 7f2b9e000640 1 -- 172.21.15.3:0/2750122545 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8031480 con 0x557fc3891d70 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.450+0000 7f2b9e000640 1 -- 172.21.15.3:0/2750122545 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f2ba8031620 con 0x557fc3891d70 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.450+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9800ee70 con 0x557fc3891d70 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.451+0000 7f2b9e000640 1 -- 172.21.15.3:0/2750122545 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ba8031820 con 0x557fc3891d70 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.451+0000 7f2bb9ffb740 10 client.25837 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.451+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557fc38a0590 con 0x557fc3891d70 2024-08-01T04:20:04.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.451+0000 7f2b9e000640 1 -- 172.21.15.3:0/2750122545 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b9800ee70 con 0x557fc3891d70 2024-08-01T04:20:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2b9e000640 1 -- 172.21.15.3:0/2750122545 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ba8004040 con 0x557fc3891d70 2024-08-01T04:20:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2bb9ffb740 10 client.25837 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2bb9ffb740 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2bb9ffb740 20 client.25837 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc38a19a0 0x557fc38a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2bb9ffb740 4 client.25837 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:04.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.459+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc37fa160 con 0x557fc38a19a0 2024-08-01T04:20:04.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.460+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc38a19a0 0x557fc38a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.461+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc38a19a0 0x557fc38a3dd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276ca4777d915cc8 server_cookie=fef931053821d3c4 in_seq=0 out_seq=0 2024-08-01T04:20:04.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.461+0000 7f2b9e000640 10 client.25837 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:04.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.463+0000 7f2b9e000640 1 -- 172.21.15.3:0/2750122545 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557fc37fa160 con 0x557fc38a19a0 2024-08-01T04:20:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.463+0000 7f2b9e000640 10 client.25837 handle_command_reply: tid=0 2024-08-01T04:20:04.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.463+0000 7f2bb9ffb740 1 client.25837 shutdown 2024-08-01T04:20:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.464+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 msgr2=0x557fc388f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.464+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc388f620 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0079330 tx=0x7f2ba8003ba0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.464+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 shutdown_connections 2024-08-01T04:20:04.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.464+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc38a19a0 0x557fc38a3dd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.465+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3891d70 0x557fc388f620 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.465+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc38874b0 0x557fc388f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.465+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2750122545 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3886b90 0x557fc388eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.465+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 >> 172.21.15.3:0/2750122545 conn(0x557fc3883120 msgr2=0x557fc3888e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.465+0000 7f2b9e000640 0 client.25837 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.466+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 shutdown_connections 2024-08-01T04:20:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.466+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2750122545 wait complete. 2024-08-01T04:20:04.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.466+0000 7f2bb9ffb740 20 client.25837 trim_cache size 0 max 16384 2024-08-01T04:20:04.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.527+0000 7f2bb9ffb740 1 Processor -- start 2024-08-01T04:20:04.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.528+0000 7f2bb9ffb740 1 -- start start 2024-08-01T04:20:04.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.528+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3886b90 0x557fc3890b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.528+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 0x557fc38a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.528+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3891d70 0x557fc38a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.528+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01c76a0 con 0x557fc3891d70 2024-08-01T04:20:04.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.529+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01c4f70 con 0x557fc3886b90 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.529+0000 7f2bb9ffb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01c2dc0 con 0x557fc38874b0 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.529+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 0x557fc38a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.529+0000 7f2bb5600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3891d70 0x557fc38a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.529+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 0x557fc38a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41072/0 (socket says 172.21.15.3:41072) 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.529+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2976053082 learned_addr learned my addr 172.21.15.3:0/2976053082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2976053082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3886b90 msgr2=0x557fc3890b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3886b90 0x557fc3890b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2976053082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3891d70 msgr2=0x557fc38a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3891d70 0x557fc38a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2ba7e00640 1 -- 172.21.15.3:0/2976053082 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc3882f50 con 0x557fc38874b0 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3891d70 0x557fc38a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:04.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.530+0000 7f2ba7e00640 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 0x557fc38a96a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b98010c90 tx=0x7f2b98004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f05534bd15371a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.531+0000 7f2ba5200640 1 -- 172.21.15.3:0/2976053082 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98037070 con 0x557fc38874b0 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.531+0000 7f2ba5200640 1 -- 172.21.15.3:0/2976053082 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b9800a3a0 con 0x557fc38874b0 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.531+0000 7f2ba5200640 1 -- 172.21.15.3:0/2976053082 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98021070 con 0x557fc38874b0 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.532+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2976053082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 msgr2=0x557fc38a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.532+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 0x557fc38a96a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b98010c90 tx=0x7f2b98004d60 comp rx=0 tx=0).stop 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.533+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2976053082 shutdown_connections 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.533+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc3891d70 0x557fc38a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.533+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc38874b0 0x557fc38a96a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.533+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2976053082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3886b90 0x557fc3890b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.533+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2976053082 >> 172.21.15.3:0/2976053082 conn(0x557fc3883120 msgr2=0x557fc3890400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:04.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.533+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2976053082 shutdown_connections 2024-08-01T04:20:04.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.534+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2976053082 wait complete. 2024-08-01T04:20:04.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.534+0000 7f2bb9ffb740 1 -- start start 2024-08-01T04:20:04.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.534+0000 7f2bb9ffb740 1 Processor -- start 2024-08-01T04:20:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.536+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3886b90 0x557fc389a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.536+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc38874b0 0x557fc38955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.537+0000 7f2bb9ffb740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3891d70 0x557fc3895ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.537+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3886b90 0x557fc389a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.537+0000 7f2bb4c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3886b90 0x557fc389a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41082/0 (socket says 172.21.15.3:41082) 2024-08-01T04:20:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2bb4c00640 1 -- 172.21.15.3:0/2921650877 learned_addr learned my addr 172.21.15.3:0/2921650877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01c3430 con 0x557fc38874b0 2024-08-01T04:20:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01c48a0 con 0x557fc3891d70 2024-08-01T04:20:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01c3b00 con 0x557fc3886b90 2024-08-01T04:20:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2bb9ffb740 10 client.0 _ll_register_callbacks cb 0x7f2ba8009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2ba7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc38874b0 0x557fc38955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.538+0000 7f2bb5600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3891d70 0x557fc3895ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2bb5600640 1 -- 172.21.15.3:0/2921650877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3886b90 msgr2=0x557fc389a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3886b90 0x557fc389a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2bb5600640 1 -- 172.21.15.3:0/2921650877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc38874b0 msgr2=0x557fc38955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc38874b0 0x557fc38955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2bb5600640 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc3884190 con 0x557fc3891d70 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2bb5600640 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3891d70 0x557fc3895ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009430 tx=0x7f2ba8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b24e8945bafe7fb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.539+0000 7f2b9e000640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:20:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.540+0000 7f2b9e000640 1 -- 172.21.15.3:0/2921650877 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8004560 con 0x557fc3891d70 2024-08-01T04:20:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.540+0000 7f2b9e000640 1 -- 172.21.15.3:0/2921650877 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f2ba80047b0 con 0x557fc3891d70 2024-08-01T04:20:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.540+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9800a3a0 con 0x557fc3891d70 2024-08-01T04:20:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.541+0000 7f2b9e000640 1 -- 172.21.15.3:0/2921650877 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ba805d050 con 0x557fc3891d70 2024-08-01T04:20:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.541+0000 7f2bb9ffb740 10 client.25853 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:04.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.541+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557fc390c1f0 con 0x557fc3891d70 2024-08-01T04:20:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.542+0000 7f2b9e000640 1 -- 172.21.15.3:0/2921650877 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba8036050 con 0x557fc3891d70 2024-08-01T04:20:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.542+0000 7f2b9e000640 1 -- 172.21.15.3:0/2921650877 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2ba8060020 con 0x557fc3891d70 2024-08-01T04:20:04.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.542+0000 7f2bb9ffb740 10 client.25853 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.543+0000 7f2bb9ffb740 10 client.25853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.543+0000 7f2bb9ffb740 20 client.25853 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.543+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc390c9c0 0x557fc390ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.543+0000 7f2bb9ffb740 4 client.25853 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:20:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.543+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2b9800a3a0 con 0x557fc390c9c0 2024-08-01T04:20:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.544+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc390c9c0 0x557fc390ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.545+0000 7f2bb4c00640 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc390c9c0 0x557fc390ede0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985ee0ec988871c server_cookie=7db3bbac4eff4e8c in_seq=0 out_seq=0 2024-08-01T04:20:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.545+0000 7f2b9e000640 10 client.25853 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.545+0000 7f2b9e000640 1 -- 172.21.15.3:0/2921650877 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f2b9800a3a0 con 0x557fc390c9c0 2024-08-01T04:20:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.545+0000 7f2b9e000640 10 client.25853 handle_command_reply: tid=0 2024-08-01T04:20:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.545+0000 7f2bb9ffb740 1 client.25853 shutdown 2024-08-01T04:20:04.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.548+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3891d70 msgr2=0x557fc3895ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.548+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3891d70 0x557fc3895ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009430 tx=0x7f2ba8004ab0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 shutdown_connections 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557fc390c9c0 0x557fc390ede0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557fc3891d70 0x557fc3895ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557fc38874b0 0x557fc38955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 --2- 172.21.15.3:0/2921650877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557fc3886b90 0x557fc389a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 >> 172.21.15.3:0/2921650877 conn(0x557fc3883120 msgr2=0x557fc3888e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:04.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2b9e000640 0 client.25853 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.549+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 shutdown_connections 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.550+0000 7f2bb9ffb740 1 -- 172.21.15.3:0/2921650877 wait complete. 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.550+0000 7f2bb9ffb740 20 client.25853 trim_cache size 0 max 16384 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:20:04.552 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:20:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.551+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2b940a2f00 msgr2=0x7f2b940a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.551+0000 7f2bb7600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2b940a2f00 0x7f2b940a53c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0104ec0 tx=0x7f2b98005ec0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.551+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 msgr2=0x7f2bb01085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:04.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.551+0000 7f2bb7600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 0x7f2bb01085c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000b450 tx=0x7f2ba000b910 comp rx=0 tx=0).stop 2024-08-01T04:20:04.556 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 shutdown_connections 2024-08-01T04:20:04.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bb0112760 0x7f2bb01039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bb0076d70 0x7f2bb0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2b940a2f00 0x7f2b940a53c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 --2- 172.21.15.3:0/3209562114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb0076450 0x7f2bb01085c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 >> 172.21.15.3:0/3209562114 conn(0x7f2bb00ff7c0 msgr2=0x7f2bb0101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:04.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.556+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 shutdown_connections 2024-08-01T04:20:04.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:04.559+0000 7f2bb7600640 1 -- 172.21.15.3:0/3209562114 wait complete. 2024-08-01T04:20:05.864 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:20:05.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:20:05.979 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:05 smithi003 ceph-mon[32262]: pgmap v117: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 123 MiB/s wr, 16.99k op/s 2024-08-01T04:20:06.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:05 smithi163 ceph-mon[37383]: pgmap v117: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 123 MiB/s wr, 16.99k op/s 2024-08-01T04:20:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:05 smithi160 ceph-mon[37224]: pgmap v117: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 123 MiB/s wr, 16.99k op/s 2024-08-01T04:20:06.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.274+0000 7f2495800640 1 -- 172.21.15.3:0/3142135801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 msgr2=0x7f2490114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.274+0000 7f2495800640 1 --2- 172.21.15.3:0/3142135801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f2490114de0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f247c009900 tx=0x7f247c02a410 comp rx=0 tx=0).stop 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 -- 172.21.15.3:0/3142135801 shutdown_connections 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 --2- 172.21.15.3:0/3142135801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f2490114de0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 --2- 172.21.15.3:0/3142135801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108b30 0x7f249010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 --2- 172.21.15.3:0/3142135801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108180 0x7f2490108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 -- 172.21.15.3:0/3142135801 >> 172.21.15.3:0/3142135801 conn(0x7f24900778d0 msgr2=0x7f2490079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:06.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 -- 172.21.15.3:0/3142135801 shutdown_connections 2024-08-01T04:20:06.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.275+0000 7f2495800640 1 -- 172.21.15.3:0/3142135801 wait complete. 2024-08-01T04:20:06.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.276+0000 7f2495800640 1 Processor -- start 2024-08-01T04:20:06.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.277+0000 7f2495800640 1 -- start start 2024-08-01T04:20:06.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f2495800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108180 0x7f249019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f2495800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108b30 0x7f249019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f2495800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f24901a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f2495800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2490118270 con 0x7f2490108180 2024-08-01T04:20:06.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f2495800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f24901180f0 con 0x7f2490108b30 2024-08-01T04:20:06.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f2495800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24901183f0 con 0x7f249010d260 2024-08-01T04:20:06.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f248e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108b30 0x7f249019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f248e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108b30 0x7f249019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51110/0 (socket says 172.21.15.3:51110) 2024-08-01T04:20:06.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f248e000640 1 -- 172.21.15.3:0/2029310053 learned_addr learned my addr 172.21.15.3:0/2029310053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:06.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.278+0000 7f248f400640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f24901a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248f400640 1 -- 172.21.15.3:0/2029310053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108b30 msgr2=0x7f249019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248ea00640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108180 0x7f249019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248f400640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108b30 0x7f249019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248f400640 1 -- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108180 msgr2=0x7f249019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248f400640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108180 0x7f249019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248f400640 1 -- 172.21.15.3:0/2029310053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24901a3680 con 0x7f249010d260 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.279+0000 7f248ea00640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108180 0x7f249019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:06.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.280+0000 7f248f400640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f24901a2fa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f247c009850 tx=0x7f247c003ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:06.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.280+0000 7f2487400640 1 -- 172.21.15.3:0/2029310053 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c03f070 con 0x7f249010d260 2024-08-01T04:20:06.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.280+0000 7f2487400640 1 -- 172.21.15.3:0/2029310053 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f247c031a20 con 0x7f249010d260 2024-08-01T04:20:06.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.280+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24901a3910 con 0x7f249010d260 2024-08-01T04:20:06.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.281+0000 7f2487400640 1 -- 172.21.15.3:0/2029310053 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c035030 con 0x7f249010d260 2024-08-01T04:20:06.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.281+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24901a3dc0 con 0x7f249010d260 2024-08-01T04:20:06.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.289+0000 7f2487400640 1 -- 172.21.15.3:0/2029310053 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f247c042070 con 0x7f249010d260 2024-08-01T04:20:06.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.291+0000 7f24983cd740 1 Processor -- start 2024-08-01T04:20:06.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.292+0000 7f24983cd740 1 -- start start 2024-08-01T04:20:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.293+0000 7f2487400640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24740a2d40 0x7f24740a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.293+0000 7f2487400640 1 -- 172.21.15.3:0/2029310053 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f247c0efa50 con 0x7f249010d260 2024-08-01T04:20:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.294+0000 7f24983cd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296df8d0 0x556a296dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.294+0000 7f24983cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.294+0000 7f24983cd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 0x556a296ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.295+0000 7f24983cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556a29644860 con 0x556a296e01f0 2024-08-01T04:20:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.295+0000 7f248f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 0x556a296ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.295+0000 7f248f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 0x556a296ecea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51114/0 (socket says 172.21.15.3:51114) 2024-08-01T04:20:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.295+0000 7f248f400640 1 -- 172.21.15.3:0/3463303698 learned_addr learned my addr 172.21.15.3:0/3463303698 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.295+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556a295dc0d0 con 0x556a296eaab0 2024-08-01T04:20:06.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.295+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556a29640820 con 0x556a296df8d0 2024-08-01T04:20:06.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.296+0000 7f248ea00640 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296df8d0 0x556a296dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.296+0000 7f248f400640 1 -- 172.21.15.3:0/3463303698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296df8d0 msgr2=0x556a296dfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.296+0000 7f248e000640 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.296+0000 7f248f400640 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296df8d0 0x556a296dfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.296+0000 7f248f400640 1 -- 172.21.15.3:0/3463303698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 msgr2=0x556a296ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.297+0000 7f248f400640 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.298+0000 7f248f400640 1 -- 172.21.15.3:0/3463303698 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a295e2d80 con 0x556a296eaab0 2024-08-01T04:20:06.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.298+0000 7f248e000640 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.298+0000 7f248e000640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24740a2d40 0x7f24740a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.298+0000 7f248f400640 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 0x556a296ecea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f247c005d00 tx=0x7f247c0bba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=124e1879c119d5b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:06.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.299+0000 7f247be00640 1 -- 172.21.15.3:0/3463303698 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0bbf30 con 0x556a296eaab0 2024-08-01T04:20:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.299+0000 7f247be00640 1 -- 172.21.15.3:0/3463303698 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f247c0c02e0 con 0x556a296eaab0 2024-08-01T04:20:06.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.299+0000 7f247be00640 1 -- 172.21.15.3:0/3463303698 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0bc190 con 0x556a296eaab0 2024-08-01T04:20:06.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.300+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 msgr2=0x556a296ecea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.300+0000 7f24983cd740 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 0x556a296ecea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f247c005d00 tx=0x7f247c0bba70 comp rx=0 tx=0).stop 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.301+0000 7f248e000640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24740a2d40 0x7f24740a5200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2480005f60 tx=0x7f2480005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.301+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 shutdown_connections 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.301+0000 7f24983cd740 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296eaab0 0x556a296ecea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.301+0000 7f24983cd740 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.301+0000 7f24983cd740 1 --2- 172.21.15.3:0/3463303698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296df8d0 0x556a296dfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.301+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 >> 172.21.15.3:0/3463303698 conn(0x556a296dbe60 msgr2=0x556a296dc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.302+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 shutdown_connections 2024-08-01T04:20:06.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.302+0000 7f24983cd740 1 -- 172.21.15.3:0/3463303698 wait complete. 2024-08-01T04:20:06.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.303+0000 7f24983cd740 1 -- start start 2024-08-01T04:20:06.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.303+0000 7f24983cd740 1 Processor -- start 2024-08-01T04:20:06.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296df8d0 0x556a296e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556a296ed440 con 0x556a296e01f0 2024-08-01T04:20:06.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556a295ff6f0 con 0x556a296df8d0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x556a296ed5c0 con 0x556a296eaab0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f24983cd740 10 client.0 _ll_register_callbacks cb 0x556a295eb4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.306+0000 7f248ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296df8d0 0x556a296e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296df8d0 0x556a296e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51120/0 (socket says 172.21.15.3:51120) 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248ea00640 1 -- 172.21.15.3:0/3262620777 learned_addr learned my addr 172.21.15.3:0/3262620777 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248f400640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248e000640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248f400640 1 -- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296df8d0 msgr2=0x556a296e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248f400640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296df8d0 0x556a296e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248f400640 1 -- 172.21.15.3:0/3262620777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 msgr2=0x556a296e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248f400640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296e7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.307+0000 7f248f400640 1 -- 172.21.15.3:0/3262620777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a293b6ba0 con 0x556a296eaab0 2024-08-01T04:20:06.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.309+0000 7f2478c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:06.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.309+0000 7f248f400640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e8370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x556a296e0810 tx=0x7f247c0b8840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8abc80fd108119fc server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.309+0000 7f2478c00640 1 -- 172.21.15.3:0/3262620777 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0c4ba0 con 0x556a296eaab0 2024-08-01T04:20:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.309+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f247c0c02e0 con 0x556a296eaab0 2024-08-01T04:20:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.309+0000 7f2478c00640 1 -- 172.21.15.3:0/3262620777 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f247c0d7070 con 0x556a296eaab0 2024-08-01T04:20:06.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.310+0000 7f2478c00640 1 -- 172.21.15.3:0/3262620777 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f247c0c4ba0 con 0x556a296eaab0 2024-08-01T04:20:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.311+0000 7f24983cd740 10 client.25855 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.312+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556a296ed440 con 0x556a296eaab0 2024-08-01T04:20:06.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.313+0000 7f2478c00640 1 -- 172.21.15.3:0/3262620777 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f247c0c02e0 con 0x556a296eaab0 2024-08-01T04:20:06.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.313+0000 7f2478c00640 1 -- 172.21.15.3:0/3262620777 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f247c0bc190 con 0x556a296eaab0 2024-08-01T04:20:06.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.313+0000 7f24983cd740 10 client.25855 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.314+0000 7f24983cd740 10 client.25855 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:20:06.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.314+0000 7f24983cd740 20 client.25855 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:06.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.314+0000 7f24983cd740 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a296fa1d0 0x556a296fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.314+0000 7f24983cd740 4 client.25855 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:06.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.314+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a29651f90 con 0x556a296fa1d0 2024-08-01T04:20:06.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.318+0000 7f248ea00640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a296fa1d0 0x556a296fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.321+0000 7f248ea00640 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a296fa1d0 0x556a296fc5c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4834707434b52e31 server_cookie=bbe6b69260208bc7 in_seq=0 out_seq=0 2024-08-01T04:20:06.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.321+0000 7f2478c00640 10 client.25855 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:06.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.322+0000 7f2478c00640 1 -- 172.21.15.3:0/3262620777 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a29651f90 con 0x556a296fa1d0 2024-08-01T04:20:06.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.322+0000 7f2478c00640 10 client.25855 handle_command_reply: tid=0 2024-08-01T04:20:06.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.322+0000 7f24983cd740 1 client.25855 shutdown 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.322+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 msgr2=0x556a296e8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.322+0000 7f24983cd740 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e8370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x556a296e0810 tx=0x7f247c0b8840 comp rx=0 tx=0).stop 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 shutdown_connections 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a296fa1d0 0x556a296fc5c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f2478c00640 0 client.25855 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e8370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296e01f0 0x556a296e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 --2- 172.21.15.3:0/3262620777 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296df8d0 0x556a296e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 >> 172.21.15.3:0/3262620777 conn(0x556a296dbe60 msgr2=0x556a296e1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 shutdown_connections 2024-08-01T04:20:06.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 1 -- 172.21.15.3:0/3262620777 wait complete. 2024-08-01T04:20:06.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.323+0000 7f24983cd740 20 client.25855 trim_cache size 0 max 16384 2024-08-01T04:20:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.359+0000 7f24983cd740 1 Processor -- start 2024-08-01T04:20:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.359+0000 7f24983cd740 1 -- start start 2024-08-01T04:20:06.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.359+0000 7f24983cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a29701e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.360+0000 7f248ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a29701e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.360+0000 7f24983cd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a297023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.360+0000 7f24983cd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.361+0000 7f248e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a297023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.361+0000 7f248e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a297023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51128/0 (socket says 172.21.15.3:51128) 2024-08-01T04:20:06.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.361+0000 7f248e000640 1 -- 172.21.15.3:0/2348565670 learned_addr learned my addr 172.21.15.3:0/2348565670 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:06.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.361+0000 7f248f400640 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.361+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24901addd0 con 0x556a296df8d0 2024-08-01T04:20:06.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.362+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f24901ab6a0 con 0x556a296e01f0 2024-08-01T04:20:06.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.362+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f249019fa30 con 0x556a296eaab0 2024-08-01T04:20:06.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.363+0000 7f248ea00640 1 -- 172.21.15.3:0/2348565670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 msgr2=0x556a296e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.363+0000 7f248ea00640 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.363+0000 7f248ea00640 1 -- 172.21.15.3:0/2348565670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 msgr2=0x556a297023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.363+0000 7f248ea00640 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a297023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.363+0000 7f248ea00640 1 -- 172.21.15.3:0/2348565670 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a296fc930 con 0x556a296df8d0 2024-08-01T04:20:06.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.363+0000 7f248f400640 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:06.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.364+0000 7f248ea00640 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a29701e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556a296fcbc0 tx=0x7f2488014bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4faea24b692455 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:06.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.365+0000 7f247b400640 1 -- 172.21.15.3:0/2348565670 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488014d00 con 0x556a296df8d0 2024-08-01T04:20:06.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.365+0000 7f247b400640 1 -- 172.21.15.3:0/2348565670 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2488011e50 con 0x556a296df8d0 2024-08-01T04:20:06.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 msgr2=0x556a29701e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:06.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a29701e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556a296fcbc0 tx=0x7f2488014bd0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.365+0000 7f247b400640 1 -- 172.21.15.3:0/2348565670 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488014d00 con 0x556a296df8d0 2024-08-01T04:20:06.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 shutdown_connections 2024-08-01T04:20:06.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a297023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 --2- 172.21.15.3:0/2348565670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a29701e90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.366+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 >> 172.21.15.3:0/2348565670 conn(0x556a296dbe60 msgr2=0x556a296e9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:06.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.367+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 shutdown_connections 2024-08-01T04:20:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.367+0000 7f24983cd740 1 -- 172.21.15.3:0/2348565670 wait complete. 2024-08-01T04:20:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.368+0000 7f24983cd740 1 -- start start 2024-08-01T04:20:06.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.368+0000 7f24983cd740 1 Processor -- start 2024-08-01T04:20:06.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a296f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a296f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2490072980 con 0x556a296df8d0 2024-08-01T04:20:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2490073df0 con 0x556a296e01f0 2024-08-01T04:20:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2490073050 con 0x556a296eaab0 2024-08-01T04:20:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.374+0000 7f24983cd740 10 client.0 _ll_register_callbacks cb 0x7f247c0bc6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:06.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.376+0000 7f248ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a296f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.377+0000 7f248e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a296f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.377+0000 7f248f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.380+0000 7f248f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41136/0 (socket says 172.21.15.3:41136) 2024-08-01T04:20:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.380+0000 7f248f400640 1 -- 172.21.15.3:0/2620984171 learned_addr learned my addr 172.21.15.3:0/2620984171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:06.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.378+0000 7f248ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a296f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46184/0 (socket says 172.21.15.3:46184) 2024-08-01T04:20:06.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.381+0000 7f248f400640 1 -- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 msgr2=0x556a296f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.381+0000 7f248f400640 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a296f30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.381+0000 7f248f400640 1 -- 172.21.15.3:0/2620984171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 msgr2=0x556a296f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.381+0000 7f248f400640 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a296f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.381+0000 7f248f400640 1 -- 172.21.15.3:0/2620984171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a296e4b60 con 0x556a296eaab0 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.381+0000 7f248f400640 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e4360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f247c005d00 tx=0x7f247c0bba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b82df916c3d7906 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.382+0000 7f2478c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.382+0000 7f2478c00640 1 -- 172.21.15.3:0/2620984171 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0bb890 con 0x556a296eaab0 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.382+0000 7f2478c00640 1 -- 172.21.15.3:0/2620984171 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f247c0ba230 con 0x556a296eaab0 2024-08-01T04:20:06.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.382+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2488011e50 con 0x556a296eaab0 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.382+0000 7f2478c00640 1 -- 172.21.15.3:0/2620984171 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f247c0ca050 con 0x556a296eaab0 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.382+0000 7f24983cd740 10 client.25861 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556a296e4b60 con 0x556a296eaab0 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f2478c00640 1 -- 172.21.15.3:0/2620984171 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2488011e50 con 0x556a296eaab0 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f2478c00640 1 -- 172.21.15.3:0/2620984171 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f247c0bc190 con 0x556a296eaab0 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 10 client.25861 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 10 client.25861 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:20:06.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 20 client.25861 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a29764f20 0x556a297673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 4 client.25861 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.55, "await_for": 45.0}] 2024-08-01T04:20:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.383+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.55, "await_for": 45.0}) -- 0x556a2964c2e0 con 0x556a29764f20 2024-08-01T04:20:06.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.386+0000 7f248ea00640 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a29764f20 0x556a297673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.387+0000 7f2478c00640 10 client.25861 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:06.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:06.388+0000 7f248ea00640 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a29764f20 0x556a297673e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6d82a17783d8c0 server_cookie=cce0e19a4c76ea8 in_seq=0 out_seq=0 2024-08-01T04:20:06.910 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.826+0000 7f5b70a00640 1 -- 172.21.15.3:0/266448268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109180 msgr2=0x7f5b6c109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.827+0000 7f5b70a00640 1 --2- 172.21.15.3:0/266448268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109180 0x7f5b6c109560 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c007380 tx=0x7f5b5c02aff0 comp rx=0 tx=0).stop 2024-08-01T04:20:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 -- 172.21.15.3:0/266448268 shutdown_connections 2024-08-01T04:20:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 --2- 172.21.15.3:0/266448268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b6c10e7b0 0x7f5b6c115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 --2- 172.21.15.3:0/266448268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 --2- 172.21.15.3:0/266448268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109180 0x7f5b6c109560 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:07.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 -- 172.21.15.3:0/266448268 >> 172.21.15.3:0/266448268 conn(0x7f5b6c06e000 msgr2=0x7f5b6c06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 -- 172.21.15.3:0/266448268 shutdown_connections 2024-08-01T04:20:07.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.828+0000 7f5b70a00640 1 -- 172.21.15.3:0/266448268 wait complete. 2024-08-01T04:20:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.830+0000 7f5b70a00640 1 Processor -- start 2024-08-01T04:20:07.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.831+0000 7f5b70a00640 1 -- start start 2024-08-01T04:20:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.831+0000 7f5b70a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b6c109180 0x7f5b6c10f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.832+0000 7f5b70a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.832+0000 7f5b70a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b6c10e7b0 0x7f5b6c10fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.832+0000 7f5b70a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c118270 con 0x7f5b6c109180 2024-08-01T04:20:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.832+0000 7f5b70a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1180f0 con 0x7f5b6c10e7b0 2024-08-01T04:20:07.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.832+0000 7f5b70a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1183f0 con 0x7f5b6c109b30 2024-08-01T04:20:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.833+0000 7f5b69400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.833+0000 7f5b69400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41160/0 (socket says 172.21.15.3:41160) 2024-08-01T04:20:07.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.833+0000 7f5b69400640 1 -- 172.21.15.3:0/1839999712 learned_addr learned my addr 172.21.15.3:0/1839999712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.835+0000 7f5b69400640 1 -- 172.21.15.3:0/1839999712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b6c10e7b0 msgr2=0x7f5b6c10fc90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-01T04:20:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.835+0000 7f5b69400640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b6c10e7b0 0x7f5b6c10fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.835+0000 7f5b69400640 1 -- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b6c109180 msgr2=0x7f5b6c10f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.835+0000 7f5b69400640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b6c109180 0x7f5b6c10f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:07.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.835+0000 7f5b69400640 1 -- 172.21.15.3:0/1839999712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1a2990 con 0x7f5b6c109b30 2024-08-01T04:20:07.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.836+0000 7f5b69400640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10f750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e9d0 tx=0x7f5b5800ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.837+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800cde0 con 0x7f5b6c109b30 2024-08-01T04:20:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.837+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b580035f0 con 0x7f5b6c109b30 2024-08-01T04:20:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.837+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58010690 con 0x7f5b6c109b30 2024-08-01T04:20:07.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.837+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1a2c80 con 0x7f5b6c109b30 2024-08-01T04:20:07.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.838+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a3170 con 0x7f5b6c109b30 2024-08-01T04:20:07.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.841+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c073c70 con 0x7f5b6c109b30 2024-08-01T04:20:07.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.852+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5b58003b30 con 0x7f5b6c109b30 2024-08-01T04:20:07.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.856+0000 7f5b62a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b500a2d70 0x7f5b500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:07.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.857+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f5b580cf9a0 con 0x7f5b6c109b30 2024-08-01T04:20:07.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.857+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5b580fde70 con 0x7f5b6c109b30 2024-08-01T04:20:07.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.858+0000 7f5b69e00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b500a2d70 0x7f5b500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:07.870+0000 7f5b69e00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b500a2d70 0x7f5b500a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005b60 tx=0x7f5b5c008a60 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:08.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:07 smithi163 ceph-mon[37383]: pgmap v118: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 134 MiB/s wr, 18.66k op/s 2024-08-01T04:20:08.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:07 smithi003 ceph-mon[32262]: pgmap v118: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 134 MiB/s wr, 18.66k op/s 2024-08-01T04:20:08.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:07 smithi160 ceph-mon[37224]: pgmap v118: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 134 MiB/s wr, 18.66k op/s 2024-08-01T04:20:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.394+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b6c109560 con 0x7f5b6c109b30 2024-08-01T04:20:08.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.394+0000 7f5b62a00640 1 -- 172.21.15.3:0/1839999712 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11911 (secure 0 0 0) 0x7f5b5801f340 con 0x7f5b6c109b30 2024-08-01T04:20:08.400 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:20:08.400 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-08-01T04:18:17:181881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:18:16.489018+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.398+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b500a2d70 msgr2=0x7f5b500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.398+0000 7f5b70a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b500a2d70 0x7f5b500a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005b60 tx=0x7f5b5c008a60 comp rx=0 tx=0).stop 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.398+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 msgr2=0x7f5b6c10f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.398+0000 7f5b70a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10f750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e9d0 tx=0x7f5b5800ee90 comp rx=0 tx=0).stop 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.399+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 shutdown_connections 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.399+0000 7f5b70a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b6c10e7b0 0x7f5b6c10fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.399+0000 7f5b70a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f5b500a2d70 0x7f5b500a5230 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005b60 tx=0x7f5b5c008a60 comp rx=0 tx=0).stop 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.399+0000 7f5b70a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c109b30 0x7f5b6c10f750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.399+0000 7f5b70a00640 1 --2- 172.21.15.3:0/1839999712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b6c109180 0x7f5b6c10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:08.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.399+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 >> 172.21.15.3:0/1839999712 conn(0x7f5b6c06e000 msgr2=0x7f5b6c1137f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.404+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 shutdown_connections 2024-08-01T04:20:08.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:08.404+0000 7f5b70a00640 1 -- 172.21.15.3:0/1839999712 wait complete. 2024-08-01T04:20:09.034 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:08 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1839999712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:09.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:08 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1839999712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:09.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:08 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1839999712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:09.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:20:09.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:09 smithi003 ceph-mon[32262]: pgmap v119: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 160 MiB/s wr, 22.12k op/s 2024-08-01T04:20:10.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:09 smithi163 ceph-mon[37383]: pgmap v119: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 160 MiB/s wr, 22.12k op/s 2024-08-01T04:20:10.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:09 smithi160 ceph-mon[37224]: pgmap v119: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 160 MiB/s wr, 22.12k op/s 2024-08-01T04:20:10.407 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.176+0000 7f2853e00640 1 -- 172.21.15.3:0/1423211881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f285410dfb0 msgr2=0x7f28541103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.176+0000 7f2853e00640 1 --2- 172.21.15.3:0/1423211881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f285410dfb0 0x7f28541103a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c02a940 comp rx=0 tx=0).stop 2024-08-01T04:20:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 -- 172.21.15.3:0/1423211881 shutdown_connections 2024-08-01T04:20:11.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 --2- 172.21.15.3:0/1423211881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f285410dfb0 0x7f28541103a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 --2- 172.21.15.3:0/1423211881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2854074b60 0x7f285410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 --2- 172.21.15.3:0/1423211881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2854074240 0x7f2854074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 -- 172.21.15.3:0/1423211881 >> 172.21.15.3:0/1423211881 conn(0x7f28540fd470 msgr2=0x7f28540ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 -- 172.21.15.3:0/1423211881 shutdown_connections 2024-08-01T04:20:11.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.179+0000 7f2853e00640 1 -- 172.21.15.3:0/1423211881 wait complete. 2024-08-01T04:20:11.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.182+0000 7f2853e00640 1 Processor -- start 2024-08-01T04:20:11.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.184+0000 7f2853e00640 1 -- start start 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.185+0000 7f2853e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2854074240 0x7f28540737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.185+0000 7f2853e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2854074b60 0x7f285406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.185+0000 7f2853e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 0x7f285406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.185+0000 7f2853e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2854113ce0 con 0x7f285410dfb0 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.185+0000 7f2853e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2854113b60 con 0x7f2854074b60 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.185+0000 7f2853e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2854113e60 con 0x7f2854074240 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.186+0000 7f2853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 0x7f285406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.186+0000 7f2853400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 0x7f285406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46218/0 (socket says 172.21.15.3:46218) 2024-08-01T04:20:11.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.186+0000 7f2853400640 1 -- 172.21.15.3:0/2398261807 learned_addr learned my addr 172.21.15.3:0/2398261807 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:11.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.186+0000 7f2852a00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2854074240 0x7f28540737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.187+0000 7f2852000640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2854074b60 0x7f285406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.188+0000 7f2853400640 1 -- 172.21.15.3:0/2398261807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2854074240 msgr2=0x7f28540737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.188+0000 7f2853400640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2854074240 0x7f28540737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.188+0000 7f2853400640 1 -- 172.21.15.3:0/2398261807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2854074b60 msgr2=0x7f285406c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.188+0000 7f2853400640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2854074b60 0x7f285406c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.188+0000 7f2853400640 1 -- 172.21.15.3:0/2398261807 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285406d3d0 con 0x7f285410dfb0 2024-08-01T04:20:11.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.188+0000 7f2852000640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2854074b60 0x7f285406c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:11.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.190+0000 7f2853400640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 0x7f285406cde0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f284c005cc0 tx=0x7f284c02f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:11.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.192+0000 7f284b400640 1 -- 172.21.15.3:0/2398261807 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c034040 con 0x7f285410dfb0 2024-08-01T04:20:11.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.192+0000 7f284b400640 1 -- 172.21.15.3:0/2398261807 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284c003e50 con 0x7f285410dfb0 2024-08-01T04:20:11.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.192+0000 7f284b400640 1 -- 172.21.15.3:0/2398261807 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c0047c0 con 0x7f285410dfb0 2024-08-01T04:20:11.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.193+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285406d660 con 0x7f285410dfb0 2024-08-01T04:20:11.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.193+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2854071570 con 0x7f285410dfb0 2024-08-01T04:20:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.200+0000 7f285c276740 1 Processor -- start 2024-08-01T04:20:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.201+0000 7f284b400640 1 -- 172.21.15.3:0/2398261807 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f284c037020 con 0x7f285410dfb0 2024-08-01T04:20:11.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.201+0000 7f285c276740 1 -- start start 2024-08-01T04:20:11.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.202+0000 7f285c276740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2d56e0 0x55a82e2d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.202+0000 7f285c276740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.202+0000 7f285c276740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2e08c0 0x55a82e2e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.202+0000 7f285c276740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a82e23abf0 con 0x55a82e2e08c0 2024-08-01T04:20:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.202+0000 7f285c276740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a82e1d2b20 con 0x55a82e2d6000 2024-08-01T04:20:11.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.202+0000 7f285c276740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a82e236bb0 con 0x55a82e2d56e0 2024-08-01T04:20:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.204+0000 7f284b400640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f28380a2d70 0x7f28380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.204+0000 7f284b400640 1 -- 172.21.15.3:0/2398261807 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f284c0f03e0 con 0x7f285410dfb0 2024-08-01T04:20:11.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.205+0000 7f2852000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2d56e0 0x55a82e2d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2853400640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f28380a2d70 0x7f28380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2d56e0 0x55a82e2d5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41192/0 (socket says 172.21.15.3:41192) 2024-08-01T04:20:11.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852000640 1 -- 172.21.15.3:0/3678258704 learned_addr learned my addr 172.21.15.3:0/3678258704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:11.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852000640 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2e08c0 0x55a82e2e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852a00640 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852a00640 1 -- 172.21.15.3:0/3678258704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2d56e0 msgr2=0x55a82e2d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852a00640 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2d56e0 0x55a82e2d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852a00640 1 -- 172.21.15.3:0/3678258704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2e08c0 msgr2=0x55a82e2e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852a00640 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2e08c0 0x55a82e2e2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.206+0000 7f2852a00640 1 -- 172.21.15.3:0/3678258704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82e1d97f0 con 0x55a82e2d6000 2024-08-01T04:20:11.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.207+0000 7f2853400640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f28380a2d70 0x7f28380a5230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f284c0edcb0 tx=0x7f284c02a940 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:11.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.207+0000 7f2852a00640 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e0380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f284000bbf0 tx=0x7f28400107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15143f0cf41e3808 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:11.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.211+0000 7f283f400640 1 -- 172.21.15.3:0/3678258704 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840004030 con 0x55a82e2d6000 2024-08-01T04:20:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.211+0000 7f283f400640 1 -- 172.21.15.3:0/3678258704 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28400041d0 con 0x55a82e2d6000 2024-08-01T04:20:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.211+0000 7f283f400640 1 -- 172.21.15.3:0/3678258704 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400116a0 con 0x55a82e2d6000 2024-08-01T04:20:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.212+0000 7f285c276740 1 -- 172.21.15.3:0/3678258704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 msgr2=0x55a82e2e0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.212+0000 7f285c276740 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e0380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f284000bbf0 tx=0x7f28400107a0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.215+0000 7f285c276740 1 -- 172.21.15.3:0/3678258704 shutdown_connections 2024-08-01T04:20:11.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.215+0000 7f285c276740 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2e08c0 0x55a82e2e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.215+0000 7f285c276740 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e0380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.215+0000 7f285c276740 1 --2- 172.21.15.3:0/3678258704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2d56e0 0x55a82e2d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.215+0000 7f285c276740 1 -- 172.21.15.3:0/3678258704 >> 172.21.15.3:0/3678258704 conn(0x55a82e2d1c70 msgr2=0x55a82e2d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:11.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.215+0000 7f285c276740 1 -- 172.21.15.3:0/3678258704 shutdown_connections 2024-08-01T04:20:11.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.216+0000 7f285c276740 1 -- 172.21.15.3:0/3678258704 wait complete. 2024-08-01T04:20:11.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.216+0000 7f285c276740 1 -- start start 2024-08-01T04:20:11.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.219+0000 7f285c276740 1 Processor -- start 2024-08-01T04:20:11.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d56e0 0x55a82e2dd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 0x55a82e2ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a82e2e3490 con 0x55a82e2d6000 2024-08-01T04:20:11.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a82e1f5cf0 con 0x55a82e2d56e0 2024-08-01T04:20:11.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a82e2e3610 con 0x55a82e2e08c0 2024-08-01T04:20:11.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.223+0000 7f285c276740 10 client.0 _ll_register_callbacks cb 0x55a82e1e1e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:11.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.225+0000 7f2852000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d56e0 0x55a82e2dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.226+0000 7f2852a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 0x55a82e2ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.226+0000 7f2852a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 0x55a82e2ddd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46242/0 (socket says 172.21.15.3:46242) 2024-08-01T04:20:11.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.226+0000 7f2852a00640 1 -- 172.21.15.3:0/4049701874 learned_addr learned my addr 172.21.15.3:0/4049701874 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:11.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.227+0000 7f2852000640 1 -- 172.21.15.3:0/4049701874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 msgr2=0x55a82e2de280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.227+0000 7f2852000640 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.227+0000 7f2852000640 1 -- 172.21.15.3:0/4049701874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 msgr2=0x55a82e2ddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.227+0000 7f2852000640 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 0x55a82e2ddd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.227+0000 7f2852000640 1 -- 172.21.15.3:0/4049701874 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82dfac3c0 con 0x55a82e2d56e0 2024-08-01T04:20:11.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.227+0000 7f2852a00640 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 0x55a82e2ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:20:11.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.229+0000 7f2852000640 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d56e0 0x55a82e2dd800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2844005fd0 tx=0x7f2844012db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42359caa92d5a00 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:11.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.229+0000 7f2833e00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:20:11.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.229+0000 7f2833e00640 1 -- 172.21.15.3:0/4049701874 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400fc20 con 0x55a82e2d56e0 2024-08-01T04:20:11.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.230+0000 7f2833e00640 1 -- 172.21.15.3:0/4049701874 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f284400fdc0 con 0x55a82e2d56e0 2024-08-01T04:20:11.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.231+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28400041d0 con 0x55a82e2d56e0 2024-08-01T04:20:11.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.232+0000 7f2833e00640 1 -- 172.21.15.3:0/4049701874 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2844022050 con 0x55a82e2d56e0 2024-08-01T04:20:11.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.232+0000 7f285c276740 10 client.25883 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:11.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.232+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a82e2ef1f0 con 0x55a82e2d56e0 2024-08-01T04:20:11.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.233+0000 7f2833e00640 1 -- 172.21.15.3:0/4049701874 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28400041d0 con 0x55a82e2d56e0 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.234+0000 7f2833e00640 1 -- 172.21.15.3:0/4049701874 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f284401d460 con 0x55a82e2d56e0 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.235+0000 7f285c276740 10 client.25883 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.235+0000 7f285c276740 10 client.25883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.236+0000 7f285c276740 20 client.25883 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.236+0000 7f285c276740 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e2f06b0 0x55a82e2f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.236+0000 7f285c276740 4 client.25883 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:11.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.236+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a82e248160 con 0x55a82e2f06b0 2024-08-01T04:20:11.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.236+0000 7f2852a00640 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e2f06b0 0x55a82e2f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.237+0000 7f2852a00640 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e2f06b0 0x55a82e2f2ac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5eb0db1d5f40d6c1 server_cookie=57080d424720083 in_seq=0 out_seq=0 2024-08-01T04:20:11.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.237+0000 7f2833e00640 10 client.25883 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:20:11.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.239+0000 7f2833e00640 1 -- 172.21.15.3:0/4049701874 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a82e248160 con 0x55a82e2f06b0 2024-08-01T04:20:11.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.239+0000 7f2833e00640 10 client.25883 handle_command_reply: tid=0 2024-08-01T04:20:11.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.239+0000 7f285c276740 1 client.25883 shutdown 2024-08-01T04:20:11.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.240+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d56e0 msgr2=0x55a82e2dd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.240+0000 7f285c276740 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d56e0 0x55a82e2dd800 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2844005fd0 tx=0x7f2844012db0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.243+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 shutdown_connections 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.243+0000 7f285c276740 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e2f06b0 0x55a82e2f2ac0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.243+0000 7f285c276740 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.243+0000 7f285c276740 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d6000 0x55a82e2ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.243+0000 7f285c276740 1 --2- 172.21.15.3:0/4049701874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d56e0 0x55a82e2dd800 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.243+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 >> 172.21.15.3:0/4049701874 conn(0x55a82e2d1c70 msgr2=0x55a82e2d7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.244+0000 7f2833e00640 0 client.25883 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.244+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 shutdown_connections 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.244+0000 7f285c276740 1 -- 172.21.15.3:0/4049701874 wait complete. 2024-08-01T04:20:11.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.244+0000 7f285c276740 20 client.25883 trim_cache size 0 max 16384 2024-08-01T04:20:11.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.269+0000 7f285c276740 1 Processor -- start 2024-08-01T04:20:11.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.269+0000 7f285c276740 1 -- start start 2024-08-01T04:20:11.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.270+0000 7f285c276740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.270+0000 7f285c276740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.270+0000 7f285c276740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.270+0000 7f285c276740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28541bec00 con 0x55a82e2d56e0 2024-08-01T04:20:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.270+0000 7f285c276740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f28541bc4d0 con 0x55a82e2d6000 2024-08-01T04:20:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.270+0000 7f285c276740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2854105810 con 0x55a82e2e08c0 2024-08-01T04:20:11.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51212/0 (socket says 172.21.15.3:51212) 2024-08-01T04:20:11.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 -- 172.21.15.3:0/2738425119 learned_addr learned my addr 172.21.15.3:0/2738425119 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:11.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852000640 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 -- 172.21.15.3:0/2738425119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 msgr2=0x55a82e2f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 -- 172.21.15.3:0/2738425119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 msgr2=0x55a82e2d8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.271+0000 7f2852a00640 1 -- 172.21.15.3:0/2738425119 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82e2dbd50 con 0x55a82e2d6000 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.272+0000 7f2852000640 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.272+0000 7f2852a00640 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2f8390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f284000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3cc5ac96b96e26 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:11.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.274+0000 7f283ea00640 1 -- 172.21.15.3:0/2738425119 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284000bda0 con 0x55a82e2d6000 2024-08-01T04:20:11.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.275+0000 7f283ea00640 1 -- 172.21.15.3:0/2738425119 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840003ad0 con 0x55a82e2d6000 2024-08-01T04:20:11.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.275+0000 7f283ea00640 1 -- 172.21.15.3:0/2738425119 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28400116a0 con 0x55a82e2d6000 2024-08-01T04:20:11.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 -- 172.21.15.3:0/2738425119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 msgr2=0x55a82e2f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2f8390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f284000b390 comp rx=0 tx=0).stop 2024-08-01T04:20:11.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 -- 172.21.15.3:0/2738425119 shutdown_connections 2024-08-01T04:20:11.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2f8390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 --2- 172.21.15.3:0/2738425119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2d8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.276+0000 7f285c276740 1 -- 172.21.15.3:0/2738425119 >> 172.21.15.3:0/2738425119 conn(0x55a82e2d1c70 msgr2=0x55a82e2d8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:11.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.278+0000 7f285c276740 1 -- 172.21.15.3:0/2738425119 shutdown_connections 2024-08-01T04:20:11.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.279+0000 7f285c276740 1 -- 172.21.15.3:0/2738425119 wait complete. 2024-08-01T04:20:11.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.279+0000 7f285c276740 1 -- start start 2024-08-01T04:20:11.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.280+0000 7f285c276740 1 Processor -- start 2024-08-01T04:20:11.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.281+0000 7f285c276740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.281+0000 7f285c276740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f285c276740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f285c276740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28541ba990 con 0x55a82e2d56e0 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f285c276740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f28541bbe00 con 0x55a82e2d6000 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f285c276740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f28541bb060 con 0x55a82e2e08c0 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f2852000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f2852000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f2852000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2e41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41216/0 (socket says 172.21.15.3:41216) 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f2852000640 1 -- 172.21.15.3:0/2848735671 learned_addr learned my addr 172.21.15.3:0/2848735671 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f2852a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.282+0000 7f285c276740 10 client.0 _ll_register_callbacks cb 0x7f2844012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.283+0000 7f2852a00640 1 -- 172.21.15.3:0/2848735671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 msgr2=0x55a82e2e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.283+0000 7f2852a00640 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.283+0000 7f2852a00640 1 -- 172.21.15.3:0/2848735671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 msgr2=0x55a82e2e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.283+0000 7f2852a00640 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.283+0000 7f2852a00640 1 -- 172.21.15.3:0/2848735671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82e2e49d0 con 0x55a82e2d6000 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.284+0000 7f2852a00640 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e9110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55a82e2ccbb0 tx=0x7f2840003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d625730410b1b74e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.284+0000 7f2833e00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.285+0000 7f2833e00640 1 -- 172.21.15.3:0/2848735671 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840005120 con 0x55a82e2d6000 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.285+0000 7f2833e00640 1 -- 172.21.15.3:0/2848735671 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f284002d040 con 0x55a82e2d6000 2024-08-01T04:20:11.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.286+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2840003ad0 con 0x55a82e2d6000 2024-08-01T04:20:11.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.286+0000 7f2833e00640 1 -- 172.21.15.3:0/2848735671 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f284003f050 con 0x55a82e2d6000 2024-08-01T04:20:11.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.287+0000 7f285c276740 10 client.25889 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:11.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.287+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a82e35afa0 con 0x55a82e2d6000 2024-08-01T04:20:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.288+0000 7f2833e00640 1 -- 172.21.15.3:0/2848735671 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840003ad0 con 0x55a82e2d6000 2024-08-01T04:20:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.288+0000 7f2833e00640 1 -- 172.21.15.3:0/2848735671 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2840032860 con 0x55a82e2d6000 2024-08-01T04:20:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.288+0000 7f285c276740 10 client.25889 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.288+0000 7f285c276740 10 client.25889 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:20:11.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.288+0000 7f285c276740 20 client.25889 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.288+0000 7f285c276740 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e35b410 0x55a82e35d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.289+0000 7f285c276740 4 client.25889 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:20:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.289+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a82e242670 con 0x55a82e35b410 2024-08-01T04:20:11.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.289+0000 7f2852000640 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e35b410 0x55a82e35d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.291+0000 7f2833e00640 10 client.25889 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:20:11.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.291+0000 7f2852000640 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e35b410 0x55a82e35d800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb1ee87b67ea39ec server_cookie=7eefa8ecd4b9e5f5 in_seq=0 out_seq=0 2024-08-01T04:20:11.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.291+0000 7f2833e00640 1 -- 172.21.15.3:0/2848735671 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a82e242670 con 0x55a82e35b410 2024-08-01T04:20:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.291+0000 7f2833e00640 10 client.25889 handle_command_reply: tid=0 2024-08-01T04:20:11.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.293+0000 7f285c276740 1 client.25889 shutdown 2024-08-01T04:20:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 msgr2=0x55a82e2e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e9110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55a82e2ccbb0 tx=0x7f2840003c80 comp rx=0 tx=0).stop 2024-08-01T04:20:11.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 shutdown_connections 2024-08-01T04:20:11.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a82e35b410 0x55a82e35d800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a82e2e08c0 0x55a82e2e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a82e2d6000 0x55a82e2e9110 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 --2- 172.21.15.3:0/2848735671 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a82e2d56e0 0x55a82e2e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f2833e00640 0 client.25889 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:20:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.295+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 >> 172.21.15.3:0/2848735671 conn(0x55a82e2d1c70 msgr2=0x55a82e2df060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.296+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 shutdown_connections 2024-08-01T04:20:11.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.296+0000 7f285c276740 1 -- 172.21.15.3:0/2848735671 wait complete. 2024-08-01T04:20:11.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.297+0000 7f285c276740 20 client.25889 trim_cache size 0 max 16384 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f28380a2d70 msgr2=0x7f28380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f28380a2d70 0x7f28380a5230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f284c0edcb0 tx=0x7f284c02a940 comp rx=0 tx=0).stop 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 msgr2=0x7f285406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 0x7f285406cde0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f284c005cc0 tx=0x7f284c02f6b0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 shutdown_connections 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f285410dfb0 0x7f285406cde0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2854074b60 0x7f285406c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f28380a2d70 0x7f28380a5230 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 --2- 172.21.15.3:0/2398261807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2854074240 0x7f28540737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.298+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 >> 172.21.15.3:0/2398261807 conn(0x7f28540fd470 msgr2=0x7f28540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.299+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 shutdown_connections 2024-08-01T04:20:11.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:11.299+0000 7f2853e00640 1 -- 172.21.15.3:0/2398261807 wait complete. 2024-08-01T04:20:12.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:11 smithi163 ceph-mon[37383]: pgmap v120: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 102 MiB/s wr, 14.05k op/s 2024-08-01T04:20:12.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:11 smithi003 ceph-mon[32262]: pgmap v120: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 102 MiB/s wr, 14.05k op/s 2024-08-01T04:20:12.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:11 smithi160 ceph-mon[37224]: pgmap v120: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 102 MiB/s wr, 14.05k op/s 2024-08-01T04:20:14.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:13 smithi163 ceph-mon[37383]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 120 MiB/s wr, 16.55k op/s 2024-08-01T04:20:14.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:13 smithi003 ceph-mon[32262]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 120 MiB/s wr, 16.55k op/s 2024-08-01T04:20:14.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:13 smithi160 ceph-mon[37224]: pgmap v121: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 120 MiB/s wr, 16.55k op/s 2024-08-01T04:20:15.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:20:15.910 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:16.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:15 smithi163 ceph-mon[37383]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 173 MiB/s wr, 24.18k op/s 2024-08-01T04:20:16.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:15 smithi003 ceph-mon[32262]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 173 MiB/s wr, 24.18k op/s 2024-08-01T04:20:16.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:15 smithi160 ceph-mon[37224]: pgmap v122: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 173 MiB/s wr, 24.18k op/s 2024-08-01T04:20:16.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.693+0000 7fc0d3e00640 1 -- 172.21.15.3:0/4151456211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d4076d70 msgr2=0x7fc0d4112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.693+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/4151456211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d4076d70 0x7fc0d4112060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009900 tx=0x7fc0cc02a400 comp rx=0 tx=0).stop 2024-08-01T04:20:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.694+0000 7fc0d3e00640 1 -- 172.21.15.3:0/4151456211 shutdown_connections 2024-08-01T04:20:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.694+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/4151456211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d41125a0 0x7fc0d4114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.694+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/4151456211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d4076d70 0x7fc0d4112060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.694+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/4151456211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076450 0x7fc0d4076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.694+0000 7fc0d3e00640 1 -- 172.21.15.3:0/4151456211 >> 172.21.15.3:0/4151456211 conn(0x7fc0d40ff640 msgr2=0x7fc0d4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.697+0000 7fc0d3e00640 1 -- 172.21.15.3:0/4151456211 shutdown_connections 2024-08-01T04:20:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.697+0000 7fc0d3e00640 1 -- 172.21.15.3:0/4151456211 wait complete. 2024-08-01T04:20:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 Processor -- start 2024-08-01T04:20:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 -- start start 2024-08-01T04:20:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d4076450 0x7fc0d419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 0x7fc0d419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d41125a0 0x7fc0d41a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41181d0 con 0x7fc0d4076d70 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4118050 con 0x7fc0d41125a0 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.699+0000 7fc0d3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4118350 con 0x7fc0d4076450 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.700+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 0x7fc0d419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.700+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 0x7fc0d419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48912/0 (socket says 172.21.15.3:48912) 2024-08-01T04:20:16.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.700+0000 7fc0d2000640 1 -- 172.21.15.3:0/1292762800 learned_addr learned my addr 172.21.15.3:0/1292762800 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d4076450 0x7fc0d419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d2000640 1 -- 172.21.15.3:0/1292762800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d4076450 msgr2=0x7fc0d419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d2000640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d4076450 0x7fc0d419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d2000640 1 -- 172.21.15.3:0/1292762800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d41125a0 msgr2=0x7fc0d41a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d3400640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d41125a0 0x7fc0d41a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d2000640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d41125a0 0x7fc0d41a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.701+0000 7fc0d2000640 1 -- 172.21.15.3:0/1292762800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d41a35e0 con 0x7fc0d4076d70 2024-08-01T04:20:16.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.702+0000 7fc0d3400640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d41125a0 0x7fc0d41a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.703+0000 7fc0d2000640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 0x7fc0d419eb70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc004770 tx=0x7fc0cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.703+0000 7fc0cb400640 1 -- 172.21.15.3:0/1292762800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc03f070 con 0x7fc0d4076d70 2024-08-01T04:20:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.705+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d41a3870 con 0x7fc0d4076d70 2024-08-01T04:20:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.706+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a3ce0 con 0x7fc0d4076d70 2024-08-01T04:20:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.707+0000 7fc0cb400640 1 -- 172.21.15.3:0/1292762800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc031ca0 con 0x7fc0d4076d70 2024-08-01T04:20:16.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.707+0000 7fc0cb400640 1 -- 172.21.15.3:0/1292762800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc049060 con 0x7fc0d4076d70 2024-08-01T04:20:16.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.717+0000 7fc0dc452740 1 Processor -- start 2024-08-01T04:20:16.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.717+0000 7fc0dc452740 1 -- start start 2024-08-01T04:20:16.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.718+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 0x55f0de935ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.719+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de940380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.719+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9408c0 0x55f0de942cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0dc452740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f0de89abf0 con 0x55f0de936000 2024-08-01T04:20:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0dc452740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f0de832b20 con 0x55f0de9356e0 2024-08-01T04:20:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0dc452740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f0de896bb0 con 0x55f0de9408c0 2024-08-01T04:20:16.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.719+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 0x55f0de935ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 0x55f0de935ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:36886/0 (socket says 172.21.15.3:36886) 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0d3400640 1 -- 172.21.15.3:0/2449494235 learned_addr learned my addr 172.21.15.3:0/2449494235 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0d3400640 1 -- 172.21.15.3:0/2449494235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9408c0 msgr2=0x55f0de942cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0d3400640 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9408c0 0x55f0de942cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.720+0000 7fc0d3400640 1 -- 172.21.15.3:0/2449494235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 msgr2=0x55f0de940380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.721+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de940380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.721+0000 7fc0d3400640 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de940380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.722+0000 7fc0d3400640 1 -- 172.21.15.3:0/2449494235 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0de8397f0 con 0x55f0de9356e0 2024-08-01T04:20:16.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.723+0000 7fc0cb400640 1 -- 172.21.15.3:0/1292762800 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc0cc042070 con 0x7fc0d4076d70 2024-08-01T04:20:16.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.726+0000 7fc0d3400640 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 0x55f0de935ac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009e40 tx=0x7fc0c400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a91bdec225e4c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:16.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.727+0000 7fc0cb400640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc0b80a2f70 0x7fc0b80a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.727+0000 7fc0bfe00640 1 -- 172.21.15.3:0/2449494235 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004d20 con 0x55f0de9356e0 2024-08-01T04:20:16.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.728+0000 7fc0bfe00640 1 -- 172.21.15.3:0/2449494235 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c4005500 con 0x55f0de9356e0 2024-08-01T04:20:16.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.728+0000 7fc0bfe00640 1 -- 172.21.15.3:0/2449494235 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004d20 con 0x55f0de9356e0 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.730+0000 7fc0dc452740 1 -- 172.21.15.3:0/2449494235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 msgr2=0x55f0de935ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.730+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 0x55f0de935ac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009e40 tx=0x7fc0c400f730 comp rx=0 tx=0).stop 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.730+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc0b80a2f70 0x7fc0b80a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.728+0000 7fc0cb400640 1 -- 172.21.15.3:0/1292762800 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc0cc0ef4e0 con 0x7fc0d4076d70 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.731+0000 7fc0dc452740 1 -- 172.21.15.3:0/2449494235 shutdown_connections 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.732+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9408c0 0x55f0de942cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.732+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de940380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.732+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2449494235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9356e0 0x55f0de935ac0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.732+0000 7fc0dc452740 1 -- 172.21.15.3:0/2449494235 >> 172.21.15.3:0/2449494235 conn(0x55f0de931c70 msgr2=0x55f0de932050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:16.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.732+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc0b80a2f70 0x7fc0b80a5430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f0de932ad0 tx=0x7fc0c0008040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:16.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.735+0000 7fc0dc452740 1 -- 172.21.15.3:0/2449494235 shutdown_connections 2024-08-01T04:20:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.735+0000 7fc0dc452740 1 -- 172.21.15.3:0/2449494235 wait complete. 2024-08-01T04:20:16.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.736+0000 7fc0dc452740 1 -- start start 2024-08-01T04:20:16.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.736+0000 7fc0dc452740 1 Processor -- start 2024-08-01T04:20:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.737+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 0x55f0de93d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.737+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de936000 0x55f0de93dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.737+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de93e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.737+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 0x55f0de93d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.738+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 0x55f0de93d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41714/0 (socket says 172.21.15.3:41714) 2024-08-01T04:20:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de936000 0x55f0de93dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f0de9434e0 con 0x55f0de9408c0 2024-08-01T04:20:16.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de93e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f0de855cf0 con 0x55f0de936000 2024-08-01T04:20:16.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f0de943660 con 0x55f0de9356e0 2024-08-01T04:20:16.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0dc452740 10 client.0 _ll_register_callbacks cb 0x55f0de841e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:16.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.739+0000 7fc0d2a00640 1 -- 172.21.15.3:0/238546906 learned_addr learned my addr 172.21.15.3:0/238546906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.741+0000 7fc0d2a00640 1 -- 172.21.15.3:0/238546906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de936000 msgr2=0x55f0de93dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.741+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de936000 0x55f0de93dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.742+0000 7fc0d2a00640 1 -- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 msgr2=0x55f0de93e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.742+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de93e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.742+0000 7fc0d2a00640 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0de60c3c0 con 0x55f0de9356e0 2024-08-01T04:20:16.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.743+0000 7fc0d2000640 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de936000 0x55f0de93dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:20:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.743+0000 7fc0bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:16.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.744+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 0x55f0de93d7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ed70 tx=0x7fc0c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2b2d345c9ea5e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:16.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.744+0000 7fc0bcc00640 1 -- 172.21.15.3:0/238546906 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0025060 con 0x55f0de9356e0 2024-08-01T04:20:16.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.745+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c4005500 con 0x55f0de9356e0 2024-08-01T04:20:16.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.745+0000 7fc0bcc00640 1 -- 172.21.15.3:0/238546906 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc0c0015b50 con 0x55f0de9356e0 2024-08-01T04:20:16.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.746+0000 7fc0bcc00640 1 -- 172.21.15.3:0/238546906 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0c0025060 con 0x55f0de9356e0 2024-08-01T04:20:16.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.746+0000 7fc0dc452740 10 client.25891 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:16.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.746+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0de94f300 con 0x55f0de9356e0 2024-08-01T04:20:16.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.747+0000 7fc0bcc00640 1 -- 172.21.15.3:0/238546906 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c4005500 con 0x55f0de9356e0 2024-08-01T04:20:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.750+0000 7fc0bcc00640 1 -- 172.21.15.3:0/238546906 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc0c0012030 con 0x55f0de9356e0 2024-08-01T04:20:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.751+0000 7fc0dc452740 10 client.25891 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.751+0000 7fc0dc452740 10 client.25891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:20:16.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.751+0000 7fc0dc452740 20 client.25891 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.751+0000 7fc0dc452740 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de950710 0x55f0de952b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.751+0000 7fc0dc452740 4 client.25891 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.751+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0de8a8160 con 0x55f0de950710 2024-08-01T04:20:16.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.752+0000 7fc0d2000640 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de950710 0x55f0de952b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.756+0000 7fc0d2000640 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de950710 0x55f0de952b00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16ba6519f39de07b server_cookie=a3658f10b3938110 in_seq=0 out_seq=0 2024-08-01T04:20:16.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.756+0000 7fc0bcc00640 10 client.25891 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:20:16.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.758+0000 7fc0bcc00640 1 -- 172.21.15.3:0/238546906 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f0de8a8160 con 0x55f0de950710 2024-08-01T04:20:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.758+0000 7fc0bcc00640 10 client.25891 handle_command_reply: tid=0 2024-08-01T04:20:16.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.759+0000 7fc0dc452740 1 client.25891 shutdown 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 msgr2=0x55f0de93d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 0x55f0de93d7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ed70 tx=0x7fc0c00046c0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 shutdown_connections 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de950710 0x55f0de952b00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de93e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de936000 0x55f0de93dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 --2- 172.21.15.3:0/238546906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de9356e0 0x55f0de93d7e0 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ed70 tx=0x7fc0c00046c0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.760+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 >> 172.21.15.3:0/238546906 conn(0x55f0de931c70 msgr2=0x55f0de9379a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.761+0000 7fc0bcc00640 0 client.25891 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:20:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.761+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 shutdown_connections 2024-08-01T04:20:16.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.762+0000 7fc0dc452740 1 -- 172.21.15.3:0/238546906 wait complete. 2024-08-01T04:20:16.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.762+0000 7fc0dc452740 20 client.25891 trim_cache size 0 max 16384 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.798+0000 7fc0dc452740 1 Processor -- start 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.798+0000 7fc0dc452740 1 -- start start 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.799+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 0x55f0de9583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.799+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de958910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.799+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de930480 0x55f0de9308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.799+0000 7fc0dc452740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b4011220 con 0x55f0de9408c0 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.799+0000 7fc0dc452740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b400eaf0 con 0x55f0de930480 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.799+0000 7fc0dc452740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b400c940 con 0x55f0de936000 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.800+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 0x55f0de9583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.800+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 0x55f0de9583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:41722/0 (socket says 172.21.15.3:41722) 2024-08-01T04:20:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.800+0000 7fc0d2a00640 1 -- 172.21.15.3:0/2919837741 learned_addr learned my addr 172.21.15.3:0/2919837741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.801+0000 7fc0d3400640 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de930480 0x55f0de9308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.801+0000 7fc0d2000640 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de958910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.802+0000 7fc0d2000640 1 -- 172.21.15.3:0/2919837741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 msgr2=0x55f0de9583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:16.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.802+0000 7fc0d2000640 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 0x55f0de9583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.802+0000 7fc0d2000640 1 -- 172.21.15.3:0/2919837741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de930480 msgr2=0x55f0de9308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.802+0000 7fc0d2000640 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de930480 0x55f0de9308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.802+0000 7fc0d2000640 1 -- 172.21.15.3:0/2919837741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0de93bd30 con 0x55f0de9408c0 2024-08-01T04:20:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.802+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 0x55f0de9583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.803+0000 7fc0d2000640 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de958910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0ecb90 tx=0x7fc0cc0a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a0a5f1ad5de369 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:16.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.803+0000 7fc0bf400640 1 -- 172.21.15.3:0/2919837741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc03f040 con 0x55f0de9408c0 2024-08-01T04:20:16.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.804+0000 7fc0bf400640 1 -- 172.21.15.3:0/2919837741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc0c45d0 con 0x55f0de9408c0 2024-08-01T04:20:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.805+0000 7fc0dc452740 1 -- 172.21.15.3:0/2919837741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 msgr2=0x55f0de958910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.805+0000 7fc0bf400640 1 -- 172.21.15.3:0/2919837741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc03f040 con 0x55f0de9408c0 2024-08-01T04:20:16.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.805+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de958910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0ecb90 tx=0x7fc0cc0a6040 comp rx=0 tx=0).stop 2024-08-01T04:20:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.806+0000 7fc0dc452740 1 -- 172.21.15.3:0/2919837741 shutdown_connections 2024-08-01T04:20:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.806+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de930480 0x55f0de9308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.806+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de9408c0 0x55f0de958910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.806+0000 7fc0dc452740 1 --2- 172.21.15.3:0/2919837741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de936000 0x55f0de9583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.807+0000 7fc0dc452740 1 -- 172.21.15.3:0/2919837741 >> 172.21.15.3:0/2919837741 conn(0x55f0de931c70 msgr2=0x55f0de938b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.807+0000 7fc0dc452740 1 -- 172.21.15.3:0/2919837741 shutdown_connections 2024-08-01T04:20:16.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.807+0000 7fc0dc452740 1 -- 172.21.15.3:0/2919837741 wait complete. 2024-08-01T04:20:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.808+0000 7fc0dc452740 1 -- start start 2024-08-01T04:20:16.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.809+0000 7fc0dc452740 1 Processor -- start 2024-08-01T04:20:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.810+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de930480 0x55f0de948990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.810+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de948ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de930480 0x55f0de948990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0dc452740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 0x55f0de945960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0dc452740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b400cfb0 con 0x55f0de936000 2024-08-01T04:20:16.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0dc452740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b400e420 con 0x55f0de9408c0 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0dc452740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b400d680 con 0x55f0de930480 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de948ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0dc452740 10 client.0 _ll_register_callbacks cb 0x7fc0c000e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 0x55f0de945960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 0x55f0de945960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:36928/0 (socket says 172.21.15.3:36928) 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.811+0000 7fc0d3400640 1 -- 172.21.15.3:0/786101751 learned_addr learned my addr 172.21.15.3:0/786101751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:16.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d2000640 1 -- 172.21.15.3:0/786101751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de930480 msgr2=0x55f0de948990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:16.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d2000640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de930480 0x55f0de948990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d2000640 1 -- 172.21.15.3:0/786101751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 msgr2=0x55f0de945960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d2000640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 0x55f0de945960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d2000640 1 -- 172.21.15.3:0/786101751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0de949540 con 0x55f0de936000 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d3400640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 0x55f0de945960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.813+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de930480 0x55f0de948990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.814+0000 7fc0bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.814+0000 7fc0d2000640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de948ed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0cffd0 tx=0x7fc0cc0bb730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5702145b370614cc server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.814+0000 7fc0bcc00640 1 -- 172.21.15.3:0/786101751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc03f070 con 0x55f0de936000 2024-08-01T04:20:16.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.814+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0cc0c45d0 con 0x55f0de936000 2024-08-01T04:20:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.815+0000 7fc0bcc00640 1 -- 172.21.15.3:0/786101751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc0cc0b8110 con 0x55f0de936000 2024-08-01T04:20:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.816+0000 7fc0bcc00640 1 -- 172.21.15.3:0/786101751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0cc03f070 con 0x55f0de936000 2024-08-01T04:20:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.816+0000 7fc0dc452740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-08-01T04:20:16.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.816+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f0de9bb920 con 0x55f0de936000 2024-08-01T04:20:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0bcc00640 1 -- 172.21.15.3:0/786101751 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0cc0c45d0 con 0x55f0de936000 2024-08-01T04:20:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0bcc00640 1 -- 172.21.15.3:0/786101751 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc0cc04c060 con 0x55f0de936000 2024-08-01T04:20:16.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0dc452740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-08-01T04:20:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0dc452740 10 client.15747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:20:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0dc452740 20 client.15747 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0dc452740 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de9bbe60 0x55f0de9be280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0dc452740 4 client.15747 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:20:16.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.817+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0de8a2670 con 0x55f0de9bbe60 2024-08-01T04:20:16.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.819+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de9bbe60 0x55f0de9be280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:16.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.823+0000 7fc0d2a00640 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de9bbe60 0x55f0de9be280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f706e14d6d4b986d server_cookie=7056318594435c90 in_seq=0 out_seq=0 2024-08-01T04:20:16.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.823+0000 7fc0bcc00640 10 client.15747 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:20:16.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.823+0000 7fc0bcc00640 1 -- 172.21.15.3:0/786101751 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0de8a2670 con 0x55f0de9bbe60 2024-08-01T04:20:16.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.823+0000 7fc0bcc00640 10 client.15747 handle_command_reply: tid=0 2024-08-01T04:20:16.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.824+0000 7fc0dc452740 1 client.15747 shutdown 2024-08-01T04:20:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 msgr2=0x55f0de948ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de948ed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0cffd0 tx=0x7fc0cc0bb730 comp rx=0 tx=0).stop 2024-08-01T04:20:16.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 shutdown_connections 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55f0de9bbe60 0x55f0de9be280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f0de9408c0 0x55f0de945960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f0de936000 0x55f0de948ed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 --2- 172.21.15.3:0/786101751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f0de930480 0x55f0de948990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 >> 172.21.15.3:0/786101751 conn(0x55f0de931c70 msgr2=0x55f0de930860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.827+0000 7fc0bcc00640 0 client.15747 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.828+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 shutdown_connections 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.828+0000 7fc0dc452740 1 -- 172.21.15.3:0/786101751 wait complete. 2024-08-01T04:20:16.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.828+0000 7fc0dc452740 20 client.15747 trim_cache size 0 max 16384 2024-08-01T04:20:16.831 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:20:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc0b80a2f70 msgr2=0x7fc0b80a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc0b80a2f70 0x7fc0b80a5430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f0de932ad0 tx=0x7fc0c0008040 comp rx=0 tx=0).stop 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 msgr2=0x7fc0d419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 0x7fc0d419eb70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc004770 tx=0x7fc0cc003c10 comp rx=0 tx=0).stop 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 shutdown_connections 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0d41125a0 0x7fc0d41a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0d4076d70 0x7fc0d419eb70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc0b80a2f70 0x7fc0b80a5430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 --2- 172.21.15.3:0/1292762800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0d4076450 0x7fc0d419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:16.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.829+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 >> 172.21.15.3:0/1292762800 conn(0x7fc0d40ff640 msgr2=0x7fc0d4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.830+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 shutdown_connections 2024-08-01T04:20:16.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:16.830+0000 7fc0d3e00640 1 -- 172.21.15.3:0/1292762800 wait complete. 2024-08-01T04:20:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:17 smithi163 ceph-mon[37383]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 169 MiB/s wr, 23.53k op/s 2024-08-01T04:20:18.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:17 smithi003 ceph-mon[32262]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 169 MiB/s wr, 23.53k op/s 2024-08-01T04:20:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:17 smithi160 ceph-mon[37224]: pgmap v123: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 169 MiB/s wr, 23.53k op/s 2024-08-01T04:20:18.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:20:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.951+0000 7f2478c00640 1 -- 172.21.15.3:0/2620984171 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556a2964c2e0 con 0x556a29764f20 2024-08-01T04:20:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.951+0000 7f2478c00640 10 client.25861 handle_command_reply: tid=0 2024-08-01T04:20:18.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.951+0000 7f24983cd740 1 client.25861 shutdown 2024-08-01T04:20:18.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.952+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 msgr2=0x556a296e4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e4360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f247c005d00 tx=0x7f247c0bba70 comp rx=0 tx=0).stop 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 shutdown_connections 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x556a29764f20 0x556a297673e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556a296eaab0 0x556a296e4360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556a296e01f0 0x556a296f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 --2- 172.21.15.3:0/2620984171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556a296df8d0 0x556a296f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f2478c00640 0 client.25861 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.953+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 >> 172.21.15.3:0/2620984171 conn(0x556a296dbe60 msgr2=0x556a296e6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.954+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 shutdown_connections 2024-08-01T04:20:18.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.954+0000 7f24983cd740 1 -- 172.21.15.3:0/2620984171 wait complete. 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.954+0000 7f24983cd740 20 client.25861 trim_cache size 0 max 16384 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 27, 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 10, 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:20:18.957 INFO:teuthology.orchestra.run.smithi003.stdout: "7b84b8a1": { 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 10, 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 143.6, 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:20:18.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:18.959 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.956+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24740a2d40 msgr2=0x7f24740a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.956+0000 7f2495800640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24740a2d40 0x7f24740a5200 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2480005f60 tx=0x7f2480005f90 comp rx=0 tx=0).stop 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.956+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 msgr2=0x7f24901a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.956+0000 7f2495800640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f24901a2fa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f247c009850 tx=0x7f247c003ae0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.957+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 shutdown_connections 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.957+0000 7f2495800640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24740a2d40 0x7f24740a5200 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.957+0000 7f2495800640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f249010d260 0x7f24901a2fa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.957+0000 7f2495800640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2490108b30 0x7f249019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.957+0000 7f2495800640 1 --2- 172.21.15.3:0/2029310053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2490108180 0x7f249019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:18.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.957+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 >> 172.21.15.3:0/2029310053 conn(0x7f24900778d0 msgr2=0x7f2490079cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:18.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.960+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 shutdown_connections 2024-08-01T04:20:18.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:18.960+0000 7f2495800640 1 -- 172.21.15.3:0/2029310053 wait complete. 2024-08-01T04:20:20.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:19 smithi163 ceph-mon[37383]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 180 MiB/s wr, 25.54k op/s 2024-08-01T04:20:20.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:19 smithi003 ceph-mon[32262]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 180 MiB/s wr, 25.54k op/s 2024-08-01T04:20:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:19 smithi160 ceph-mon[37224]: pgmap v124: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 180 MiB/s wr, 25.54k op/s 2024-08-01T04:20:22.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:21 smithi003 ceph-mon[32262]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 147 MiB/s wr, 21.27k op/s 2024-08-01T04:20:22.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:21 smithi160 ceph-mon[37224]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 147 MiB/s wr, 21.27k op/s 2024-08-01T04:20:22.163 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7b84b8a1', quiesce took 39.7 seconds. Will release after: -16.150000000000002 2024-08-01T04:20:22.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:20:22.260 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:22.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:21 smithi163 ceph-mon[37383]: pgmap v125: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 147 MiB/s wr, 21.27k op/s 2024-08-01T04:20:22.968 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:22 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:20:22.968 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:22 smithi003 ceph-mon[32262]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 151 MiB/s wr, 21.98k op/s 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 -- 172.21.15.3:0/1763331041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa664108b30 msgr2=0x7fa66410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 --2- 172.21.15.3:0/1763331041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa664108b30 0x7fa66410cb30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa650009900 tx=0x7fa65002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 -- 172.21.15.3:0/1763331041 shutdown_connections 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 --2- 172.21.15.3:0/1763331041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa66410d260 0x7fa664114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 --2- 172.21.15.3:0/1763331041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa664108b30 0x7fa66410cb30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 --2- 172.21.15.3:0/1763331041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa664108180 0x7fa664108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.043+0000 7fa66a400640 1 -- 172.21.15.3:0/1763331041 >> 172.21.15.3:0/1763331041 conn(0x7fa6640778d0 msgr2=0x7fa664079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.044+0000 7fa66a400640 1 -- 172.21.15.3:0/1763331041 shutdown_connections 2024-08-01T04:20:23.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.044+0000 7fa66a400640 1 -- 172.21.15.3:0/1763331041 wait complete. 2024-08-01T04:20:23.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.045+0000 7fa66a400640 1 Processor -- start 2024-08-01T04:20:23.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.046+0000 7fa66a400640 1 -- start start 2024-08-01T04:20:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa66a400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa664108180 0x7fa664082580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa66a400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa664108b30 0x7fa66407b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa66a400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa66410d260 0x7fa66407bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa66a400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa664118330 con 0x7fa66410d260 2024-08-01T04:20:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa66a400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641181b0 con 0x7fa664108b30 2024-08-01T04:20:23.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa66a400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641184b0 con 0x7fa664108180 2024-08-01T04:20:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa66410d260 0x7fa66407bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa66410d260 0x7fa66407bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53494/0 (socket says 172.21.15.3:53494) 2024-08-01T04:20:23.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.047+0000 7fa663e00640 1 -- 172.21.15.3:0/3850081357 learned_addr learned my addr 172.21.15.3:0/3850081357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:23.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.049+0000 7fa662a00640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa664108b30 0x7fa66407b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.050+0000 7fa663400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa664108180 0x7fa664082580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.051+0000 7fa662a00640 1 -- 172.21.15.3:0/3850081357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa664108180 msgr2=0x7fa664082580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.051+0000 7fa662a00640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa664108180 0x7fa664082580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.051+0000 7fa662a00640 1 -- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa66410d260 msgr2=0x7fa66407bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.051+0000 7fa662a00640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa66410d260 0x7fa66407bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.051+0000 7fa662a00640 1 -- 172.21.15.3:0/3850081357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66407c140 con 0x7fa664108b30 2024-08-01T04:20:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.051+0000 7fa662a00640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa664108b30 0x7fa66407b610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa65002a9e0 tx=0x7fa650003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.054+0000 7fa65be00640 1 -- 172.21.15.3:0/3850081357 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65003f070 con 0x7fa664108b30 2024-08-01T04:20:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.055+0000 7fa65be00640 1 -- 172.21.15.3:0/3850081357 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65003b460 con 0x7fa664108b30 2024-08-01T04:20:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.055+0000 7fa65be00640 1 -- 172.21.15.3:0/3850081357 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650035030 con 0x7fa664108b30 2024-08-01T04:20:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.055+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66407c3d0 con 0x7fa664108b30 2024-08-01T04:20:23.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.055+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa664080280 con 0x7fa664108b30 2024-08-01T04:20:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.060+0000 7fa65be00640 1 -- 172.21.15.3:0/3850081357 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa650042070 con 0x7fa664108b30 2024-08-01T04:20:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.064+0000 7fa65be00640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6480a2e40 0x7fa6480a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.064+0000 7fa65be00640 1 -- 172.21.15.3:0/3850081357 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa6500efb50 con 0x7fa664108b30 2024-08-01T04:20:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.064+0000 7fa66ce0b740 1 Processor -- start 2024-08-01T04:20:23.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.065+0000 7fa663400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6480a2e40 0x7fa6480a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa663400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6480a2e40 0x7fa6480a5300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa654009890 tx=0x7fa654008040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:23.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 -- start start 2024-08-01T04:20:23.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5fd98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fe46d0 0x559ba5fe6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559ba5f3ebf0 con 0x559ba5fd94f0 2024-08-01T04:20:23.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x559ba5ed6b20 con 0x559ba5fd9e10 2024-08-01T04:20:23.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.067+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559ba5f3abb0 con 0x559ba5fe46d0 2024-08-01T04:20:23.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.070+0000 7fa663400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5fd98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.070+0000 7fa663e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fe46d0 0x559ba5fe6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.070+0000 7fa663400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5fd98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53508/0 (socket says 172.21.15.3:53508) 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.070+0000 7fa663400640 1 -- 172.21.15.3:0/3939840078 learned_addr learned my addr 172.21.15.3:0/3939840078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.070+0000 7fa662a00640 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.070+0000 7fa663e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fe46d0 0x559ba5fe6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:59898/0 (socket says 172.21.15.3:59898) 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.071+0000 7fa663400640 1 -- 172.21.15.3:0/3939840078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fe46d0 msgr2=0x559ba5fe6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.071+0000 7fa663400640 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fe46d0 0x559ba5fe6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.071+0000 7fa663400640 1 -- 172.21.15.3:0/3939840078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 msgr2=0x559ba5fe4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.071+0000 7fa663400640 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.071+0000 7fa663400640 1 -- 172.21.15.3:0/3939840078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba5edd7f0 con 0x559ba5fd94f0 2024-08-01T04:20:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.071+0000 7fa663400640 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5fd98d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa65400d830 tx=0x7fa654010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe30755ac613542d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.072+0000 7fa658c00640 1 -- 172.21.15.3:0/3939840078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654024070 con 0x559ba5fd94f0 2024-08-01T04:20:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.072+0000 7fa658c00640 1 -- 172.21.15.3:0/3939840078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65401f400 con 0x559ba5fd94f0 2024-08-01T04:20:23.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.072+0000 7fa658c00640 1 -- 172.21.15.3:0/3939840078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654024070 con 0x559ba5fd94f0 2024-08-01T04:20:23.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.073+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3939840078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 msgr2=0x559ba5fd98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.073+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5fd98d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa65400d830 tx=0x7fa654010000 comp rx=0 tx=0).stop 2024-08-01T04:20:23.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3939840078 shutdown_connections 2024-08-01T04:20:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fe46d0 0x559ba5fe6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3939840078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5fd98d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3939840078 >> 172.21.15.3:0/3939840078 conn(0x559ba5fd5a80 msgr2=0x559ba5fd5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:23.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3939840078 shutdown_connections 2024-08-01T04:20:23.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.074+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3939840078 wait complete. 2024-08-01T04:20:23.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.075+0000 7fa66ce0b740 1 -- start start 2024-08-01T04:20:23.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.075+0000 7fa66ce0b740 1 Processor -- start 2024-08-01T04:20:23.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 0x559ba5fe1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5fe1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559ba5fe72a0 con 0x559ba5fe46d0 2024-08-01T04:20:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x559ba5ef9cf0 con 0x559ba5fd94f0 2024-08-01T04:20:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559ba5fe7420 con 0x559ba5fd9e10 2024-08-01T04:20:23.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.076+0000 7fa66ce0b740 10 client.0 _ll_register_callbacks cb 0x559ba5ee5e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.077+0000 7fa663400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 0x559ba5fe1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.077+0000 7fa663400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 0x559ba5fe1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:42682/0 (socket says 172.21.15.3:42682) 2024-08-01T04:20:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.077+0000 7fa663400640 1 -- 172.21.15.3:0/3443497735 learned_addr learned my addr 172.21.15.3:0/3443497735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.077+0000 7fa662a00640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5fe1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa662a00640 1 -- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 msgr2=0x559ba5fe1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa662a00640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 0x559ba5fe1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa663e00640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa662a00640 1 -- 172.21.15.3:0/3443497735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 msgr2=0x559ba5fe2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa662a00640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa662a00640 1 -- 172.21.15.3:0/3443497735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba5cb03c0 con 0x559ba5fd9e10 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa663400640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 0x559ba5fe1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa662a00640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5fe1b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6500ed200 tx=0x7fa6500bbb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42082678c20c12a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:23.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.078+0000 7fa64d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:23.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.079+0000 7fa64d600640 1 -- 172.21.15.3:0/3443497735 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500bbfd0 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.079+0000 7fa64d600640 1 -- 172.21.15.3:0/3443497735 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa6500ed480 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.079+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65401f400 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.080+0000 7fa64d600640 1 -- 172.21.15.3:0/3443497735 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6500ed680 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.080+0000 7fa66ce0b740 10 client.25906 fetch_fsmap learned FSMap version 28 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.080+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ba5fe7330 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.080+0000 7fa64d600640 1 -- 172.21.15.3:0/3443497735 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65401f400 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa64d600640 1 -- 172.21.15.3:0/3443497735 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fa6500bb420 con 0x559ba5fd9e10 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa66ce0b740 10 client.25906 fetch_fsmap finished waiting for FSMap version 28 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa66ce0b740 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa66ce0b740 20 client.25906 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba5ff45c0 0x559ba5ff6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa66ce0b740 4 client.25906 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.081+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ba5f4c160 con 0x559ba5ff45c0 2024-08-01T04:20:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.082+0000 7fa663400640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba5ff45c0 0x559ba5ff6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.082+0000 7fa663400640 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba5ff45c0 0x559ba5ff6a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dfced5b1893d5f5 server_cookie=ea6656791fd8ac8c in_seq=0 out_seq=0 2024-08-01T04:20:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.082+0000 7fa64d600640 10 client.25906 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.083+0000 7fa64d600640 1 -- 172.21.15.3:0/3443497735 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ba5f4c160 con 0x559ba5ff45c0 2024-08-01T04:20:23.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.083+0000 7fa64d600640 10 client.25906 handle_command_reply: tid=0 2024-08-01T04:20:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.083+0000 7fa66ce0b740 1 client.25906 shutdown 2024-08-01T04:20:23.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.084+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 msgr2=0x559ba5fe1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.084+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5fe1b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6500ed200 tx=0x7fa6500bbb10 comp rx=0 tx=0).stop 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 shutdown_connections 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba5ff45c0 0x559ba5ff6a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5fe1b50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3443497735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd94f0 0x559ba5fe1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 >> 172.21.15.3:0/3443497735 conn(0x559ba5fd5a80 msgr2=0x559ba5fd4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:23.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa64d600640 0 client.25906 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 shutdown_connections 2024-08-01T04:20:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3443497735 wait complete. 2024-08-01T04:20:23.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.085+0000 7fa66ce0b740 20 client.25906 trim_cache size 0 max 16384 2024-08-01T04:20:23.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.112+0000 7fa66ce0b740 1 Processor -- start 2024-08-01T04:20:23.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.112+0000 7fa66ce0b740 1 -- start start 2024-08-01T04:20:23.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.113+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5ffc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.113+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5ffc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.113+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fe46d0 0x559ba5fde870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.113+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa644011250 con 0x559ba5fd94f0 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.113+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa64400eb20 con 0x559ba5fe46d0 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.113+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa64400c970 con 0x559ba5fd9e10 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa663400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5ffc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5ffc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa663e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fe46d0 0x559ba5fde870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5ffc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:59912/0 (socket says 172.21.15.3:59912) 2024-08-01T04:20:23.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 -- 172.21.15.3:0/2193292700 learned_addr learned my addr 172.21.15.3:0/2193292700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 -- 172.21.15.3:0/2193292700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fe46d0 msgr2=0x559ba5fde870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fe46d0 0x559ba5fde870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 -- 172.21.15.3:0/2193292700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 msgr2=0x559ba5ffc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5ffc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.114+0000 7fa662a00640 1 -- 172.21.15.3:0/2193292700 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ba5fd6c10 con 0x559ba5fd9e10 2024-08-01T04:20:23.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.115+0000 7fa662a00640 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5ffc830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa650009850 tx=0x7fa650005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a276b258b3958aeb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:23.121 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.118+0000 7fa64fe00640 1 -- 172.21.15.3:0/2193292700 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65009e070 con 0x559ba5fd9e10 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.118+0000 7fa64fe00640 1 -- 172.21.15.3:0/2193292700 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6500c4c40 con 0x559ba5fd9e10 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.118+0000 7fa64fe00640 1 -- 172.21.15.3:0/2193292700 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500b6b40 con 0x559ba5fd9e10 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.119+0000 7fa66ce0b740 1 -- 172.21.15.3:0/2193292700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 msgr2=0x559ba5ffc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.119+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5ffc830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa650009850 tx=0x7fa650005c90 comp rx=0 tx=0).stop 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 -- 172.21.15.3:0/2193292700 shutdown_connections 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fe46d0 0x559ba5fde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd9e10 0x559ba5ffc830 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/2193292700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fd94f0 0x559ba5ffc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 -- 172.21.15.3:0/2193292700 >> 172.21.15.3:0/2193292700 conn(0x559ba5fd5a80 msgr2=0x559ba5fe3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 -- 172.21.15.3:0/2193292700 shutdown_connections 2024-08-01T04:20:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.120+0000 7fa66ce0b740 1 -- 172.21.15.3:0/2193292700 wait complete. 2024-08-01T04:20:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.121+0000 7fa66ce0b740 1 -- start start 2024-08-01T04:20:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.121+0000 7fa66ce0b740 1 Processor -- start 2024-08-01T04:20:23.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.121+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd94f0 0x559ba5fe8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.121+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.121+0000 7fa66ce0b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa64400cfe0 con 0x559ba5fe46d0 2024-08-01T04:20:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa64400e450 con 0x559ba5fd9e10 2024-08-01T04:20:23.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa66ce0b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa64400d6b0 con 0x559ba5fd94f0 2024-08-01T04:20:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa66ce0b740 10 client.0 _ll_register_callbacks cb 0x7fa6500b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa663400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd94f0 0x559ba5fe8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa662a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.122+0000 7fa663e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa662a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:42700/0 (socket says 172.21.15.3:42700) 2024-08-01T04:20:23.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa662a00640 1 -- 172.21.15.3:0/3737273764 learned_addr learned my addr 172.21.15.3:0/3737273764 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa663400640 1 -- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 msgr2=0x559ba5fe89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa663400640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa663400640 1 -- 172.21.15.3:0/3737273764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 msgr2=0x559ba5fe8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa663400640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa663400640 1 -- 172.21.15.3:0/3737273764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ba605e310 con 0x559ba5fd94f0 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.123+0000 7fa662a00640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:20:23.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.124+0000 7fa663e00640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.124+0000 7fa663400640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd94f0 0x559ba5fe8480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x559ba5ff7290 tx=0x7fa65401fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ab0dde358ad43f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.124+0000 7fa64d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.125+0000 7fa64d600640 1 -- 172.21.15.3:0/3737273764 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654024040 con 0x559ba5fd94f0 2024-08-01T04:20:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.125+0000 7fa64d600640 1 -- 172.21.15.3:0/3737273764 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa6540149b0 con 0x559ba5fd94f0 2024-08-01T04:20:23.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.126+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6500c4c40 con 0x559ba5fd94f0 2024-08-01T04:20:23.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.126+0000 7fa64d600640 1 -- 172.21.15.3:0/3737273764 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa65401fe60 con 0x559ba5fd94f0 2024-08-01T04:20:23.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.127+0000 7fa66ce0b740 10 client.25912 fetch_fsmap learned FSMap version 28 2024-08-01T04:20:23.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.127+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ba605f5f0 con 0x559ba5fd94f0 2024-08-01T04:20:23.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.127+0000 7fa64d600640 1 -- 172.21.15.3:0/3737273764 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6500c4c40 con 0x559ba5fd94f0 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa64d600640 1 -- 172.21.15.3:0/3737273764 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7fa654013070 con 0x559ba5fd94f0 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa66ce0b740 10 client.25912 fetch_fsmap finished waiting for FSMap version 28 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa66ce0b740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa66ce0b740 20 client.25912 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba605fbd0 0x559ba6061fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa66ce0b740 4 client.25912 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:20:23.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ba5f46670 con 0x559ba605fbd0 2024-08-01T04:20:23.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.128+0000 7fa662a00640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba605fbd0 0x559ba6061fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:23.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.130+0000 7fa662a00640 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba605fbd0 0x559ba6061fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05bc78371a2b89a server_cookie=e866eaea57f96629 in_seq=0 out_seq=0 2024-08-01T04:20:23.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.130+0000 7fa64d600640 10 client.25912 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.130+0000 7fa64d600640 1 -- 172.21.15.3:0/3737273764 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ba5f46670 con 0x559ba605fbd0 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.130+0000 7fa64d600640 10 client.25912 handle_command_reply: tid=0 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.130+0000 7fa66ce0b740 1 client.25912 shutdown 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.131+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd94f0 msgr2=0x559ba5fe8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.131+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd94f0 0x559ba5fe8480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x559ba5ff7290 tx=0x7fa65401fc40 comp rx=0 tx=0).stop 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 shutdown_connections 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559ba605fbd0 0x559ba6061fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559ba5fe46d0 0x559ba5fe8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559ba5fd9e10 0x559ba5fe89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa66ce0b740 1 --2- 172.21.15.3:0/3737273764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ba5fd94f0 0x559ba5fe8480 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 >> 172.21.15.3:0/3737273764 conn(0x559ba5fd5a80 msgr2=0x559ba5fd43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:23.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.132+0000 7fa64d600640 0 client.25912 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.133+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 shutdown_connections 2024-08-01T04:20:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.133+0000 7fa66ce0b740 1 -- 172.21.15.3:0/3737273764 wait complete. 2024-08-01T04:20:23.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.133+0000 7fa66ce0b740 20 client.25912 trim_cache size 0 max 16384 2024-08-01T04:20:23.136 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:20:23.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.135+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6480a2e40 msgr2=0x7fa6480a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.135+0000 7fa66a400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6480a2e40 0x7fa6480a5300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa654009890 tx=0x7fa654008040 comp rx=0 tx=0).stop 2024-08-01T04:20:23.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.135+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa664108b30 msgr2=0x7fa66407b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:23.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.135+0000 7fa66a400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa664108b30 0x7fa66407b610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa65002a9e0 tx=0x7fa650003cd0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 shutdown_connections 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa66410d260 0x7fa66407bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa664108b30 0x7fa66407b610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa6480a2e40 0x7fa6480a5300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 --2- 172.21.15.3:0/3850081357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa664108180 0x7fa664082580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 >> 172.21.15.3:0/3850081357 conn(0x7fa6640778d0 msgr2=0x7fa664078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:23.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 shutdown_connections 2024-08-01T04:20:23.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:23.136+0000 7fa66a400640 1 -- 172.21.15.3:0/3850081357 wait complete. 2024-08-01T04:20:23.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:22 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:20:23.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:22 smithi160 ceph-mon[37224]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 151 MiB/s wr, 21.98k op/s 2024-08-01T04:20:23.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:22 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:20:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:22 smithi163 ceph-mon[37383]: pgmap v126: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 151 MiB/s wr, 21.98k op/s 2024-08-01T04:20:25.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:25 smithi163 ceph-mon[37383]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 150 MiB/s wr, 22.45k op/s 2024-08-01T04:20:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:25 smithi003 ceph-mon[32262]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 150 MiB/s wr, 22.45k op/s 2024-08-01T04:20:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:25 smithi160 ceph-mon[37224]: pgmap v127: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 150 MiB/s wr, 22.45k op/s 2024-08-01T04:20:27.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:27 smithi163 ceph-mon[37383]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 88 MiB/s wr, 13.56k op/s 2024-08-01T04:20:27.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:27 smithi003 ceph-mon[32262]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 88 MiB/s wr, 13.56k op/s 2024-08-01T04:20:27.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:27 smithi160 ceph-mon[37224]: pgmap v128: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 88 MiB/s wr, 13.56k op/s 2024-08-01T04:20:29.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:29 smithi160 ceph-mon[37224]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 MiB/s wr, 12.83k op/s 2024-08-01T04:20:29.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:29 smithi163 ceph-mon[37383]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 MiB/s wr, 12.83k op/s 2024-08-01T04:20:29.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:29 smithi003 ceph-mon[32262]: pgmap v129: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 MiB/s wr, 12.83k op/s 2024-08-01T04:20:31.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:31 smithi160 ceph-mon[37224]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 MiB/s wr, 8.20k op/s 2024-08-01T04:20:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:31 smithi163 ceph-mon[37383]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 MiB/s wr, 8.20k op/s 2024-08-01T04:20:31.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:31 smithi003 ceph-mon[32262]: pgmap v130: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 MiB/s wr, 8.20k op/s 2024-08-01T04:20:33.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:33 smithi003 ceph-mon[32262]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 MiB/s wr, 7.00k op/s 2024-08-01T04:20:33.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:33 smithi160 ceph-mon[37224]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 MiB/s wr, 7.00k op/s 2024-08-01T04:20:33.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:33 smithi163 ceph-mon[37383]: pgmap v131: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 MiB/s wr, 7.00k op/s 2024-08-01T04:20:35.275 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:35 smithi003 ceph-mon[32262]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 32 MiB/s wr, 5.16k op/s 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 -- 172.21.15.3:0/1304733494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 msgr2=0x7fd3e4107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/1304733494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 0x7fd3e4107ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009930 tx=0x7fd3d402a420 comp rx=0 tx=0).stop 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 -- 172.21.15.3:0/1304733494 shutdown_connections 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/1304733494 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 0x7fd3e410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/1304733494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/1304733494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 0x7fd3e4107ad0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 -- 172.21.15.3:0/1304733494 >> 172.21.15.3:0/1304733494 conn(0x7fd3e40fd470 msgr2=0x7fd3e40ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 -- 172.21.15.3:0/1304733494 shutdown_connections 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.281+0000 7fd3e3e00640 1 -- 172.21.15.3:0/1304733494 wait complete. 2024-08-01T04:20:35.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.282+0000 7fd3e3e00640 1 Processor -- start 2024-08-01T04:20:35.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.282+0000 7fd3e3e00640 1 -- start start 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 0x7fd3e4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 0x7fd3e419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111b40 con 0x7fd3e4102090 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e41119c0 con 0x7fd3e41016f0 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111cc0 con 0x7fd3e41076f0 2024-08-01T04:20:35.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:56696/0 (socket says 172.21.15.3:56696) 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e2a00640 1 -- 172.21.15.3:0/2817656500 learned_addr learned my addr 172.21.15.3:0/2817656500 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e2000640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 0x7fd3e4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.283+0000 7fd3e3400640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 0x7fd3e419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2a00640 1 -- 172.21.15.3:0/2817656500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 msgr2=0x7fd3e419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2a00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 0x7fd3e419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2a00640 1 -- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 msgr2=0x7fd3e4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2a00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 0x7fd3e4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2a00640 1 -- 172.21.15.3:0/2817656500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e419cfa0 con 0x7fd3e41016f0 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2000640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 0x7fd3e4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:35.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.284+0000 7fd3e2a00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4197ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3d402a940 tx=0x7fd3d4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.285+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d403f070 con 0x7fd3e41016f0 2024-08-01T04:20:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.285+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4031c90 con 0x7fd3e41016f0 2024-08-01T04:20:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.285+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4035030 con 0x7fd3e41016f0 2024-08-01T04:20:35.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.285+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e419d230 con 0x7fd3e41016f0 2024-08-01T04:20:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.285+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e419d6c0 con 0x7fd3e41016f0 2024-08-01T04:20:35.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.287+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e4101b50 con 0x7fd3e41016f0 2024-08-01T04:20:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.289+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd3d4042070 con 0x7fd3e41016f0 2024-08-01T04:20:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.291+0000 7fd3db400640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd3c80a2d70 0x7fd3c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.291+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fd3d40efd80 con 0x7fd3e41016f0 2024-08-01T04:20:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.291+0000 7fd3e2000640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd3c80a2d70 0x7fd3c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.292+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd3d40bc900 con 0x7fd3e41016f0 2024-08-01T04:20:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.292+0000 7fd3e2000640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd3c80a2d70 0x7fd3c80a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40ffa40 tx=0x7fd3d0005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:35.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:35 smithi160 ceph-mon[37224]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 32 MiB/s wr, 5.16k op/s 2024-08-01T04:20:35.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.529+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3e4109c60 con 0x7fd3e41016f0 2024-08-01T04:20:35.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.530+0000 7fd3db400640 1 -- 172.21.15.3:0/2817656500 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11909 (secure 0 0 0) 0x7fd3d40bc200 con 0x7fd3e41016f0 2024-08-01T04:20:35.533 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:20:35.533 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-08-01T04:20:21:797564+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:21.493533+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24506,24520,24548]},"id":1}]} 2024-08-01T04:20:35.533 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd3c80a2d70 msgr2=0x7fd3c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd3c80a2d70 0x7fd3c80a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40ffa40 tx=0x7fd3d0005f90 comp rx=0 tx=0).stop 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 msgr2=0x7fd3e4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4197ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3d402a940 tx=0x7fd3d4003b20 comp rx=0 tx=0).stop 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 shutdown_connections 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd3c80a2d70 0x7fd3c80a5230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3e41076f0 0x7fd3e419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3e4102090 0x7fd3e4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 --2- 172.21.15.3:0/2817656500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3e41016f0 0x7fd3e4197ff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 >> 172.21.15.3:0/2817656500 conn(0x7fd3e40fd470 msgr2=0x7fd3e40fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.534+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 shutdown_connections 2024-08-01T04:20:35.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:35.535+0000 7fd3e3e00640 1 -- 172.21.15.3:0/2817656500 wait complete. 2024-08-01T04:20:35.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:35 smithi163 ceph-mon[37383]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 32 MiB/s wr, 5.16k op/s 2024-08-01T04:20:36.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:36 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2817656500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:36.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:36 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2817656500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:36.536 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db --set-id 7b84b8a1 --release --await' 2024-08-01T04:20:36.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db --set-id 7b84b8a1 --release --await 2024-08-01T04:20:36.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:36 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2817656500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:36.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:20:36.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:20:37.000 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:37.345 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:37 smithi003 ceph-mon[32262]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.7 MiB/s wr, 1.42k op/s 2024-08-01T04:20:37.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:37 smithi160 ceph-mon[37224]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.7 MiB/s wr, 1.42k op/s 2024-08-01T04:20:37.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:37 smithi163 ceph-mon[37383]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.7 MiB/s wr, 1.42k op/s 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 -- 172.21.15.3:0/520793061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 msgr2=0x7fa9ec10e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 --2- 172.21.15.3:0/520793061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 0x7fa9ec10e170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009930 tx=0x7fa9dc02a440 comp rx=0 tx=0).stop 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 -- 172.21.15.3:0/520793061 shutdown_connections 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 --2- 172.21.15.3:0/520793061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 0x7fa9ec114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 --2- 172.21.15.3:0/520793061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 --2- 172.21.15.3:0/520793061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 0x7fa9ec10e170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 -- 172.21.15.3:0/520793061 >> 172.21.15.3:0/520793061 conn(0x7fa9ec0776b0 msgr2=0x7fa9ec079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 -- 172.21.15.3:0/520793061 shutdown_connections 2024-08-01T04:20:37.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.835+0000 7fa9f2600640 1 -- 172.21.15.3:0/520793061 wait complete. 2024-08-01T04:20:37.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.836+0000 7fa9f2600640 1 Processor -- start 2024-08-01T04:20:37.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 -- start start 2024-08-01T04:20:37.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 0x7fa9ec19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 0x7fa9ec1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec1180e0 con 0x7fa9ec10dd90 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec117f60 con 0x7fa9ec1086b0 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.837+0000 7fa9f2600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec118260 con 0x7fa9ec107d50 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43902/0 (socket says 172.21.15.3:43902) 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 -- 172.21.15.3:0/210001077 learned_addr learned my addr 172.21.15.3:0/210001077 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:37.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 0x7fa9ec19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 0x7fa9ec1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 -- 172.21.15.3:0/210001077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 msgr2=0x7fa9ec19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 0x7fa9ec19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 -- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 msgr2=0x7fa9ec1a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 0x7fa9ec1a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9f1200640 1 -- 172.21.15.3:0/210001077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec1a3570 con 0x7fa9ec107d50 2024-08-01T04:20:37.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.838+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 0x7fa9ec19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:20:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.839+0000 7fa9f1200640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec19e5c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc02a960 tx=0x7fa9dc003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.839+0000 7fa9ea000640 1 -- 172.21.15.3:0/210001077 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc03f070 con 0x7fa9ec107d50 2024-08-01T04:20:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.839+0000 7fa9ea000640 1 -- 172.21.15.3:0/210001077 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc031c90 con 0x7fa9ec107d50 2024-08-01T04:20:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.839+0000 7fa9ea000640 1 -- 172.21.15.3:0/210001077 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc035030 con 0x7fa9ec107d50 2024-08-01T04:20:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.839+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec1a3800 con 0x7fa9ec107d50 2024-08-01T04:20:37.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.839+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec1ab140 con 0x7fa9ec107d50 2024-08-01T04:20:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.843+0000 7fa9f66c1740 1 Processor -- start 2024-08-01T04:20:37.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 -- start start 2024-08-01T04:20:37.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9ea000640 1 -- 172.21.15.3:0/210001077 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa9dc042070 con 0x7fa9ec107d50 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 0x5573d499e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d49a93d0 0x5573d49ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5573d4903e30 con 0x5573d499eb10 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5573d489bd40 con 0x5573d49a93d0 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.844+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5573d48ffdf0 con 0x5573d499e1f0 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 0x5573d499e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 0x5573d499e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43908/0 (socket says 172.21.15.3:43908) 2024-08-01T04:20:37.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1c00640 1 -- 172.21.15.3:0/4163238241 learned_addr learned my addr 172.21.15.3:0/4163238241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:37.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d49a93d0 0x5573d49ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1c00640 1 -- 172.21.15.3:0/4163238241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d49a93d0 msgr2=0x5573d49ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d49a93d0 0x5573d49ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1c00640 1 -- 172.21.15.3:0/4163238241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 msgr2=0x5573d49a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.845+0000 7fa9f1200640 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9f1c00640 1 -- 172.21.15.3:0/4163238241 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d4613580 con 0x5573d499e1f0 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9f1200640 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9ea000640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa9d00a2cf0 0x7fa9d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 0x5573d499e5d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ef0 tx=0x7fa9e40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc032ecaf9e035b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:37.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9ea000640 1 -- 172.21.15.3:0/210001077 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa9dc0ef9e0 con 0x7fa9ec107d50 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9e2000640 1 -- 172.21.15.3:0/4163238241 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4003b70 con 0x5573d499e1f0 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9e2000640 1 -- 172.21.15.3:0/4163238241 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4003d10 con 0x5573d499e1f0 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.846+0000 7fa9e2000640 1 -- 172.21.15.3:0/4163238241 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4012a80 con 0x5573d499e1f0 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.847+0000 7fa9f1200640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa9d00a2cf0 0x7fa9d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 -- 172.21.15.3:0/4163238241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 msgr2=0x5573d499e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f1200640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa9d00a2cf0 0x7fa9d00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0ed1a0 tx=0x7fa9dc0f4040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 0x5573d499e5d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009ef0 tx=0x7fa9e40117b0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 -- 172.21.15.3:0/4163238241 shutdown_connections 2024-08-01T04:20:37.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d49a93d0 0x5573d49ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/4163238241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499e1f0 0x5573d499e5d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 -- 172.21.15.3:0/4163238241 >> 172.21.15.3:0/4163238241 conn(0x5573d499a780 msgr2=0x5573d499ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 -- 172.21.15.3:0/4163238241 shutdown_connections 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.848+0000 7fa9f66c1740 1 -- 172.21.15.3:0/4163238241 wait complete. 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.849+0000 7fa9f66c1740 1 -- start start 2024-08-01T04:20:37.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.849+0000 7fa9f66c1740 1 Processor -- start 2024-08-01T04:20:37.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499e1f0 0x5573d49a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 0x5573d49a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5573d49abfa0 con 0x5573d499e1f0 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5573d48bee70 con 0x5573d499eb10 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5573d49ac120 con 0x5573d49a93d0 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f66c1740 10 client.0 _ll_register_callbacks cb 0x5573d48adf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9e3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 0x5573d49a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9e3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 0x5573d49a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:56732/0 (socket says 172.21.15.3:56732) 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9e3e00640 1 -- 172.21.15.3:0/2954514966 learned_addr learned my addr 172.21.15.3:0/2954514966 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:37.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.850+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499e1f0 0x5573d49a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9e3e00640 1 -- 172.21.15.3:0/2954514966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 msgr2=0x5573d49a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9e3e00640 1 -- 172.21.15.3:0/2954514966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499e1f0 msgr2=0x5573d49a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499e1f0 0x5573d49a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9e3e00640 1 -- 172.21.15.3:0/2954514966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d46dd960 con 0x5573d499eb10 2024-08-01T04:20:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499e1f0 0x5573d49a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 0x5573d49a67e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4012d20 tx=0x7fa9d400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37f8aee46f94bd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.851+0000 7fa9daa00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:20:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9daa00640 1 -- 172.21.15.3:0/2954514966 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4012110 con 0x5573d499eb10 2024-08-01T04:20:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e4003d10 con 0x5573d499eb10 2024-08-01T04:20:37.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9daa00640 1 -- 172.21.15.3:0/2954514966 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa9d4004050 con 0x5573d499eb10 2024-08-01T04:20:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9daa00640 1 -- 172.21.15.3:0/2954514966 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9d40122b0 con 0x5573d499eb10 2024-08-01T04:20:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9f66c1740 10 client.25937 fetch_fsmap learned FSMap version 29 2024-08-01T04:20:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573d49b8070 con 0x5573d499eb10 2024-08-01T04:20:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.852+0000 7fa9daa00640 1 -- 172.21.15.3:0/2954514966 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4003d10 con 0x5573d499eb10 2024-08-01T04:20:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9daa00640 1 -- 172.21.15.3:0/2954514966 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa9d4010430 con 0x5573d499eb10 2024-08-01T04:20:37.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f66c1740 10 client.25937 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f66c1740 10 client.25937 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f66c1740 20 client.25937 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d49bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f66c1740 4 client.25937 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573d4917850 con 0x5573d49b94d0 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.853+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d49bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.854+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d49bb920 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62eeefe0ddfc713 server_cookie=55e53c2e22104e00 in_seq=0 out_seq=0 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.854+0000 7fa9daa00640 10 client.25937 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9daa00640 1 -- 172.21.15.3:0/2954514966 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5573d4917850 con 0x5573d49b94d0 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9daa00640 10 client.25937 handle_command_reply: tid=0 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 client.25937 shutdown 2024-08-01T04:20:37.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 msgr2=0x5573d49a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 0x5573d49a67e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4012d20 tx=0x7fa9d400f790 comp rx=0 tx=0).stop 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 shutdown_connections 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d49bb920 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9daa00640 0 client.25937 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499eb10 0x5573d49a67e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/2954514966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499e1f0 0x5573d49a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.855+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 >> 172.21.15.3:0/2954514966 conn(0x5573d499a780 msgr2=0x5573d49a04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.856+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 shutdown_connections 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.856+0000 7fa9f66c1740 1 -- 172.21.15.3:0/2954514966 wait complete. 2024-08-01T04:20:37.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.856+0000 7fa9f66c1740 20 client.25937 trim_cache size 0 max 16384 2024-08-01T04:20:37.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.876+0000 7fa9f66c1740 1 Processor -- start 2024-08-01T04:20:37.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 -- start start 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49c1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49c1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c8011140 con 0x5573d499eb10 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c800ea10 con 0x5573d499e1f0 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c800c860 con 0x5573d49a93d0 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43928/0 (socket says 172.21.15.3:43928) 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1868284479 learned_addr learned my addr 172.21.15.3:0/1868284479 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.877+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1868284479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 msgr2=0x5573d49a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1868284479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 msgr2=0x5573d49c1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49c1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1868284479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d49b7ea0 con 0x5573d49a93d0 2024-08-01T04:20:37.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.878+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49c1770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4012550 tx=0x7fa9e4009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f32e8b5d2044d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.879+0000 7fa9e1600640 1 -- 172.21.15.3:0/1868284479 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4012a10 con 0x5573d49a93d0 2024-08-01T04:20:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.879+0000 7fa9e1600640 1 -- 172.21.15.3:0/1868284479 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4012bb0 con 0x5573d49a93d0 2024-08-01T04:20:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.879+0000 7fa9e1600640 1 -- 172.21.15.3:0/1868284479 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4005740 con 0x5573d49a93d0 2024-08-01T04:20:37.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.879+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1868284479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 msgr2=0x5573d49c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.879+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49c1770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4012550 tx=0x7fa9e4009500 comp rx=0 tx=0).stop 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1868284479 shutdown_connections 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d49a93d0 0x5573d49c1770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d499eb10 0x5573d49c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1868284479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1868284479 >> 172.21.15.3:0/1868284479 conn(0x5573d499a780 msgr2=0x5573d49a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1868284479 shutdown_connections 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1868284479 wait complete. 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.880+0000 7fa9f66c1740 1 -- start start 2024-08-01T04:20:37.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 Processor -- start 2024-08-01T04:20:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499eb10 0x5573d49acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 0x5573d49ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c800ced0 con 0x5573d49a93d0 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c800e340 con 0x5573d499e1f0 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c800d5a0 con 0x5573d499eb10 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f66c1740 10 client.0 _ll_register_callbacks cb 0x7fa9d4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.881+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 0x5573d49ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 0x5573d49ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36122/0 (socket says 172.21.15.3:36122) 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1257040982 learned_addr learned my addr 172.21.15.3:0/1257040982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1257040982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499eb10 msgr2=0x5573d49acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499eb10 0x5573d49acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 msgr2=0x5573d49b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:37.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.882+0000 7fa9f1c00640 1 -- 172.21.15.3:0/1257040982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d49adcb0 con 0x5573d49a93d0 2024-08-01T04:20:37.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.883+0000 7fa9f1c00640 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 0x5573d49ad480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4012720 tx=0x7fa9e4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c5a8dc848bff46 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:37.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.883+0000 7fa9daa00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:20:37.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.884+0000 7fa9daa00640 1 -- 172.21.15.3:0/1257040982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40053c0 con 0x5573d49a93d0 2024-08-01T04:20:37.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.884+0000 7fa9daa00640 1 -- 172.21.15.3:0/1257040982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa9e4005560 con 0x5573d49a93d0 2024-08-01T04:20:37.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.884+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e4012bb0 con 0x5573d49a93d0 2024-08-01T04:20:37.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.884+0000 7fa9daa00640 1 -- 172.21.15.3:0/1257040982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9e4005760 con 0x5573d49a93d0 2024-08-01T04:20:37.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9f66c1740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-08-01T04:20:37.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573d4a242e0 con 0x5573d49a93d0 2024-08-01T04:20:37.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9daa00640 1 -- 172.21.15.3:0/1257040982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4012bb0 con 0x5573d49a93d0 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9daa00640 1 -- 172.21.15.3:0/1257040982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa9e403c020 con 0x5573d49a93d0 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9f66c1740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9f66c1740 10 client.15783 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.885+0000 7fa9f66c1740 20 client.15783 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.886+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d4a265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.886+0000 7fa9f66c1740 4 client.15783 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7b84b8a1", "release": true, "await": true}] 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.886+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7b84b8a1", "release": true, "await": true}) -- 0x5573d49113a0 con 0x5573d49b94d0 2024-08-01T04:20:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.887+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d4a265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.887+0000 7fa9e3e00640 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d4a265d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe53c84a63e0d6a server_cookie=daf773876ba92cc3 in_seq=0 out_seq=0 2024-08-01T04:20:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.887+0000 7fa9daa00640 10 client.15783 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.909+0000 7fa9daa00640 1 -- 172.21.15.3:0/1257040982 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5573d49113a0 con 0x5573d49b94d0 2024-08-01T04:20:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.909+0000 7fa9daa00640 10 client.15783 handle_command_reply: tid=0 2024-08-01T04:20:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.912+0000 7fa9f66c1740 1 client.15783 shutdown 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 msgr2=0x5573d49ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 0x5573d49ad480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4012720 tx=0x7fa9e4005fd0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 shutdown_connections 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5573d49b94d0 0x5573d4a265d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5573d49a93d0 0x5573d49ad480 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5573d499eb10 0x5573d49acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 --2- 172.21.15.3:0/1257040982 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5573d499e1f0 0x5573d49b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9daa00640 0 client.15783 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.919+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 >> 172.21.15.3:0/1257040982 conn(0x5573d499a780 msgr2=0x5573d49b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.924+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 shutdown_connections 2024-08-01T04:20:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.924+0000 7fa9f66c1740 1 -- 172.21.15.3:0/1257040982 wait complete. 2024-08-01T04:20:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.924+0000 7fa9f66c1740 20 client.15783 trim_cache size 0 max 16384 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 29, 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 12, 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "7b84b8a1": { 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 12, 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:20:37.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 143.6, 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:37.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:37.929 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:20:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa9d00a2cf0 msgr2=0x7fa9d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa9d00a2cf0 0x7fa9d00a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0ed1a0 tx=0x7fa9dc0f4040 comp rx=0 tx=0).stop 2024-08-01T04:20:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 msgr2=0x7fa9ec19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec19e5c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc02a960 tx=0x7fa9dc003b20 comp rx=0 tx=0).stop 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 shutdown_connections 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9ec10dd90 0x7fa9ec1a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9ec1086b0 0x7fa9ec19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa9d00a2cf0 0x7fa9d00a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 --2- 172.21.15.3:0/210001077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9ec107d50 0x7fa9ec19e5c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 >> 172.21.15.3:0/210001077 conn(0x7fa9ec0776b0 msgr2=0x7fa9ec078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 shutdown_connections 2024-08-01T04:20:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:37.930+0000 7fa9f2600640 1 -- 172.21.15.3:0/210001077 wait complete. 2024-08-01T04:20:37.962 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:38.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:38 smithi003 ceph-mon[32262]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:20:38.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:38 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:20:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:38 smithi160 ceph-mon[37224]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:20:38.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:38 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:20:38.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:38 smithi163 ceph-mon[37383]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-01T04:20:38.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:38 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:20:39.104 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7b84b8a1', total seconds elapsed: 56.7 2024-08-01T04:20:39.105 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.55 2024-08-01T04:20:39.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:20:39.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:39 smithi003 ceph-mon[32262]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-08-01T04:20:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:39 smithi160 ceph-mon[37224]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-08-01T04:20:39.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:39 smithi163 ceph-mon[37383]: pgmap v134: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.9 MiB/s wr, 337 op/s 2024-08-01T04:20:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 -- 172.21.15.3:0/2034367981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c9010df90 msgr2=0x7f2c90110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 --2- 172.21.15.3:0/2034367981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c9010df90 0x7f2c90110380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009ca0 tx=0x7f2c8802a980 comp rx=0 tx=0).stop 2024-08-01T04:20:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 -- 172.21.15.3:0/2034367981 shutdown_connections 2024-08-01T04:20:39.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 --2- 172.21.15.3:0/2034367981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c9010df90 0x7f2c90110380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 --2- 172.21.15.3:0/2034367981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c90074c10 0x7f2c9010da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 --2- 172.21.15.3:0/2034367981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c900742f0 0x7f2c900746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 -- 172.21.15.3:0/2034367981 >> 172.21.15.3:0/2034367981 conn(0x7f2c900fd470 msgr2=0x7f2c900ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 -- 172.21.15.3:0/2034367981 shutdown_connections 2024-08-01T04:20:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.872+0000 7f2c95a00640 1 -- 172.21.15.3:0/2034367981 wait complete. 2024-08-01T04:20:39.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.874+0000 7f2c95a00640 1 Processor -- start 2024-08-01T04:20:39.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 -- start start 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 0x7f2c901b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c90074c10 0x7f2c901b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9010df90 0x7f2c901aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9006c200 con 0x7f2c900742f0 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9006c080 con 0x7f2c9010df90 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c95a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9006c380 con 0x7f2c90074c10 2024-08-01T04:20:39.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c8f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c90074c10 0x7f2c901b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 0x7f2c901b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 0x7f2c901b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36140/0 (socket says 172.21.15.3:36140) 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.877+0000 7f2c8fe00640 1 -- 172.21.15.3:0/3081076057 learned_addr learned my addr 172.21.15.3:0/3081076057 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.878+0000 7f2c8fe00640 1 -- 172.21.15.3:0/3081076057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c90074c10 msgr2=0x7f2c901b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.878+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c90074c10 0x7f2c901b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.878+0000 7f2c95000640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9010df90 0x7f2c901aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.878+0000 7f2c8fe00640 1 -- 172.21.15.3:0/3081076057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9010df90 msgr2=0x7f2c901aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.879+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9010df90 0x7f2c901aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.879+0000 7f2c8fe00640 1 -- 172.21.15.3:0/3081076057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c901ab660 con 0x7f2c900742f0 2024-08-01T04:20:39.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.879+0000 7f2c95000640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9010df90 0x7f2c901aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:39.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.880+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 0x7f2c901b0ce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d950 tx=0x7f2c8000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.881+0000 7f2c8cc00640 1 -- 172.21.15.3:0/3081076057 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80017030 con 0x7f2c900742f0 2024-08-01T04:20:39.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.881+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c901ab950 con 0x7f2c900742f0 2024-08-01T04:20:39.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.882+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901077e0 con 0x7f2c900742f0 2024-08-01T04:20:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.883+0000 7f2c8cc00640 1 -- 172.21.15.3:0/3081076057 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c80004020 con 0x7f2c900742f0 2024-08-01T04:20:39.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.883+0000 7f2c8cc00640 1 -- 172.21.15.3:0/3081076057 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800054b0 con 0x7f2c900742f0 2024-08-01T04:20:39.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.891+0000 7f2c8cc00640 1 -- 172.21.15.3:0/3081076057 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2c800041d0 con 0x7f2c900742f0 2024-08-01T04:20:39.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.893+0000 7f2c99a3d740 1 Processor -- start 2024-08-01T04:20:39.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.894+0000 7f2c99a3d740 1 -- start start 2024-08-01T04:20:39.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.897+0000 7f2c8cc00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2c740a2e40 0x7f2c740a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.898+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x55652449ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.898+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.898+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.898+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5565243b67e0 con 0x5565244a6830 2024-08-01T04:20:39.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.898+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x556524401ac0 con 0x55652449b650 2024-08-01T04:20:39.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.898+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5565243998c0 con 0x55652449bf70 2024-08-01T04:20:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.900+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x55652449ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.900+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x55652449ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:56796/0 (socket says 172.21.15.3:56796) 2024-08-01T04:20:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.900+0000 7f2c8fe00640 1 -- 172.21.15.3:0/2725769838 learned_addr learned my addr 172.21.15.3:0/2725769838 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.900+0000 7f2c8cc00640 1 -- 172.21.15.3:0/3081076057 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f2c800ceef0 con 0x7f2c900742f0 2024-08-01T04:20:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.900+0000 7f2c95000640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2c740a2e40 0x7f2c740a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.901+0000 7f2c95000640 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.901+0000 7f2c8f400640 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.903+0000 7f2c8f400640 1 -- 172.21.15.3:0/2725769838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 msgr2=0x55652449ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:39.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.903+0000 7f2c8f400640 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x55652449ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.903+0000 7f2c8f400640 1 -- 172.21.15.3:0/2725769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 msgr2=0x5565244a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.903+0000 7f2c8f400640 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.903+0000 7f2c8f400640 1 -- 172.21.15.3:0/2725769838 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565243a0760 con 0x55652449bf70 2024-08-01T04:20:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.903+0000 7f2c95000640 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.905+0000 7f2c95000640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2c740a2e40 0x7f2c740a5300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x556524498f60 tx=0x7f2c88005d80 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.905+0000 7f2c8f400640 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a62f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c901ac5b0 tx=0x7f2c7c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c600dde6add073b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.906+0000 7f2c85600640 1 -- 172.21.15.3:0/2725769838 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c005020 con 0x55652449bf70 2024-08-01T04:20:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.906+0000 7f2c85600640 1 -- 172.21.15.3:0/2725769838 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c0047a0 con 0x55652449bf70 2024-08-01T04:20:39.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.906+0000 7f2c85600640 1 -- 172.21.15.3:0/2725769838 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c018800 con 0x55652449bf70 2024-08-01T04:20:39.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 -- 172.21.15.3:0/2725769838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 msgr2=0x5565244a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a62f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c901ac5b0 tx=0x7f2c7c00f3c0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 -- 172.21.15.3:0/2725769838 shutdown_connections 2024-08-01T04:20:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a62f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/2725769838 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x55652449ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 -- 172.21.15.3:0/2725769838 >> 172.21.15.3:0/2725769838 conn(0x556524497be0 msgr2=0x556524497fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:39.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.910+0000 7f2c99a3d740 1 -- 172.21.15.3:0/2725769838 shutdown_connections 2024-08-01T04:20:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.911+0000 7f2c99a3d740 1 -- 172.21.15.3:0/2725769838 wait complete. 2024-08-01T04:20:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.911+0000 7f2c99a3d740 1 -- start start 2024-08-01T04:20:39.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.912+0000 7f2c99a3d740 1 Processor -- start 2024-08-01T04:20:39.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.915+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.916+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.916+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.918+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5565243bcae0 con 0x5565244a6830 2024-08-01T04:20:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.919+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5565243fda80 con 0x55652449b650 2024-08-01T04:20:39.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.919+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5565244a94a0 con 0x55652449bf70 2024-08-01T04:20:39.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.919+0000 7f2c99a3d740 10 client.0 _ll_register_callbacks cb 0x5565243be190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:39.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.919+0000 7f2c8f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.919+0000 7f2c95000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.920+0000 7f2c95000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36164/0 (socket says 172.21.15.3:36164) 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.920+0000 7f2c95000640 1 -- 172.21.15.3:0/199756824 learned_addr learned my addr 172.21.15.3:0/199756824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.921+0000 7f2c95000640 1 -- 172.21.15.3:0/199756824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 msgr2=0x5565244a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.921+0000 7f2c95000640 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.921+0000 7f2c95000640 1 -- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 msgr2=0x5565244a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.921+0000 7f2c95000640 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.921+0000 7f2c95000640 1 -- 172.21.15.3:0/199756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565241736a0 con 0x5565244a6830 2024-08-01T04:20:39.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.922+0000 7f2c8f400640 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:39.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.924+0000 7f2c95000640 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a41b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2c88035000 tx=0x7f2c880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39d5e8cf2e24130 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:39.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.924+0000 7f2c7a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:20:39.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.926+0000 7f2c7a000640 1 -- 172.21.15.3:0/199756824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88034040 con 0x5565244a6830 2024-08-01T04:20:39.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.926+0000 7f2c7a000640 1 -- 172.21.15.3:0/199756824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f2c88046930 con 0x5565244a6830 2024-08-01T04:20:39.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.927+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c0047a0 con 0x5565244a6830 2024-08-01T04:20:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.929+0000 7f2c7a000640 1 -- 172.21.15.3:0/199756824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c88046b30 con 0x5565244a6830 2024-08-01T04:20:39.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.929+0000 7f2c99a3d740 10 client.15792 fetch_fsmap learned FSMap version 29 2024-08-01T04:20:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.929+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565244b4fe0 con 0x5565244a6830 2024-08-01T04:20:39.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.929+0000 7f2c7a000640 1 -- 172.21.15.3:0/199756824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c0047a0 con 0x5565244a6830 2024-08-01T04:20:39.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c7a000640 1 -- 172.21.15.3:0/199756824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2c8804d070 con 0x5565244a6830 2024-08-01T04:20:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c99a3d740 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:20:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c99a3d740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:39.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c99a3d740 20 client.15792 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565244b6540 0x5565244b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c99a3d740 4 client.15792 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.931+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55652440f030 con 0x5565244b6540 2024-08-01T04:20:39.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.933+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565244b6540 0x5565244b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.934+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565244b6540 0x5565244b89e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc24e387e31bbf34 server_cookie=cc46a433b2753369 in_seq=0 out_seq=0 2024-08-01T04:20:39.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.934+0000 7f2c7a000640 10 client.15792 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.936+0000 7f2c7a000640 1 -- 172.21.15.3:0/199756824 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55652440f030 con 0x5565244b6540 2024-08-01T04:20:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.936+0000 7f2c7a000640 10 client.15792 handle_command_reply: tid=0 2024-08-01T04:20:39.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.936+0000 7f2c99a3d740 1 client.15792 shutdown 2024-08-01T04:20:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 msgr2=0x5565244a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a41b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2c88035000 tx=0x7f2c880095c0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 shutdown_connections 2024-08-01T04:20:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565244b6540 0x5565244b89e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244a6830 0x5565244a41b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55652449bf70 0x5565244a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/199756824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c7a000640 0 client.15792 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:39.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.937+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 >> 172.21.15.3:0/199756824 conn(0x556524497be0 msgr2=0x55652449d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:39.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.938+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 shutdown_connections 2024-08-01T04:20:39.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.938+0000 7f2c99a3d740 1 -- 172.21.15.3:0/199756824 wait complete. 2024-08-01T04:20:39.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.939+0000 7f2c99a3d740 20 client.15792 trim_cache size 0 max 16384 2024-08-01T04:20:39.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.984+0000 7f2c99a3d740 1 Processor -- start 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 -- start start 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 0x55652449e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 0x5565244be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70011290 con 0x55652449b650 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000eb60 con 0x55652449bf70 2024-08-01T04:20:39.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.985+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000c970 con 0x5565244a6830 2024-08-01T04:20:39.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.986+0000 7f2c8f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 0x5565244be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.986+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 0x55652449e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.986+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 0x55652449e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36174/0 (socket says 172.21.15.3:36174) 2024-08-01T04:20:39.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.986+0000 7f2c8fe00640 1 -- 172.21.15.3:0/3824028508 learned_addr learned my addr 172.21.15.3:0/3824028508 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:39.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.986+0000 7f2c8f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 0x5565244be2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2024-08-01T04:20:39.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c95000640 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c8f400640 1 -- 172.21.15.3:0/3824028508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 msgr2=0x5565244be7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c8f400640 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244be7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c8f400640 1 -- 172.21.15.3:0/3824028508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 msgr2=0x55652449e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c8f400640 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 0x55652449e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c8f400640 1 -- 172.21.15.3:0/3824028508 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565244a2700 con 0x55652449bf70 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 0x55652449e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:20:39.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.987+0000 7f2c95000640 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244be7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:39.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.990+0000 7f2c8f400640 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 0x5565244be2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5565244a4d70 tx=0x7f2c7c00a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e87b5fca55d2e3b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:39.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.992+0000 7f2c84c00640 1 -- 172.21.15.3:0/3824028508 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00fdd0 con 0x55652449bf70 2024-08-01T04:20:39.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.992+0000 7f2c84c00640 1 -- 172.21.15.3:0/3824028508 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c005520 con 0x55652449bf70 2024-08-01T04:20:39.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.992+0000 7f2c84c00640 1 -- 172.21.15.3:0/3824028508 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c018440 con 0x55652449bf70 2024-08-01T04:20:39.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3824028508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 msgr2=0x5565244be2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 0x5565244be2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5565244a4d70 tx=0x7f2c7c00a7a0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3824028508 shutdown_connections 2024-08-01T04:20:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244be7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449bf70 0x5565244be2b0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5565244a4d70 tx=0x7f2c7c00a7a0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3824028508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55652449b650 0x55652449e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.993+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3824028508 >> 172.21.15.3:0/3824028508 conn(0x556524497be0 msgr2=0x5565244a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.995+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3824028508 shutdown_connections 2024-08-01T04:20:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.995+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3824028508 wait complete. 2024-08-01T04:20:39.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.996+0000 7f2c99a3d740 1 -- start start 2024-08-01T04:20:39.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.996+0000 7f2c99a3d740 1 Processor -- start 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244af200 0x5565244ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000d020 con 0x5565244af200 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000e490 con 0x55652449b650 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7000d6f0 con 0x5565244a6830 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c99a3d740 10 client.0 _ll_register_callbacks cb 0x7f2c88009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.997+0000 7f2c8f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.998+0000 7f2c8f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43988/0 (socket says 172.21.15.3:43988) 2024-08-01T04:20:39.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.998+0000 7f2c8f400640 1 -- 172.21.15.3:0/3395375546 learned_addr learned my addr 172.21.15.3:0/3395375546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:40.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.998+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:40.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.999+0000 7f2c95000640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244af200 0x5565244ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.999+0000 7f2c8f400640 1 -- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 msgr2=0x5565244ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.999+0000 7f2c8f400640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.999+0000 7f2c8f400640 1 -- 172.21.15.3:0/3395375546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244af200 msgr2=0x5565244ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.999+0000 7f2c8f400640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244af200 0x5565244ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:39.999+0000 7f2c8f400640 1 -- 172.21.15.3:0/3395375546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565244abf80 con 0x5565244a6830 2024-08-01T04:20:40.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.002+0000 7f2c8f400640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244aecc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c01a040 tx=0x7f2c7c00a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3781ca037632a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:40.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.002+0000 7f2c7a000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:20:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.003+0000 7f2c7a000640 1 -- 172.21.15.3:0/3395375546 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c032070 con 0x5565244a6830 2024-08-01T04:20:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.003+0000 7f2c7a000640 1 -- 172.21.15.3:0/3395375546 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f2c7c005240 con 0x5565244a6830 2024-08-01T04:20:40.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.004+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c005520 con 0x5565244a6830 2024-08-01T04:20:40.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.005+0000 7f2c7a000640 1 -- 172.21.15.3:0/3395375546 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c03c050 con 0x5565244a6830 2024-08-01T04:20:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.005+0000 7f2c99a3d740 10 client.25942 fetch_fsmap learned FSMap version 29 2024-08-01T04:20:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.005+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556524521d60 con 0x5565244a6830 2024-08-01T04:20:40.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.006+0000 7f2c7a000640 1 -- 172.21.15.3:0/3395375546 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c005520 con 0x5565244a6830 2024-08-01T04:20:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.008+0000 7f2c7a000640 1 -- 172.21.15.3:0/3395375546 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2c7c018440 con 0x5565244a6830 2024-08-01T04:20:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.008+0000 7f2c99a3d740 10 client.25942 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:20:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.008+0000 7f2c99a3d740 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.008+0000 7f2c99a3d740 20 client.25942 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:40.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.008+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565245220e0 0x5565245244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.009+0000 7f2c99a3d740 4 client.25942 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:20:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.009+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556524409540 con 0x5565245220e0 2024-08-01T04:20:40.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.009+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565245220e0 0x5565245244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.011+0000 7f2c8fe00640 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565245220e0 0x5565245244d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f20857e83b692d1c server_cookie=98e927528286fdbd in_seq=0 out_seq=0 2024-08-01T04:20:40.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.011+0000 7f2c7a000640 10 client.25942 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.013+0000 7f2c7a000640 1 -- 172.21.15.3:0/3395375546 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556524409540 con 0x5565245220e0 2024-08-01T04:20:40.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.013+0000 7f2c7a000640 10 client.25942 handle_command_reply: tid=0 2024-08-01T04:20:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.013+0000 7f2c99a3d740 1 client.25942 shutdown 2024-08-01T04:20:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 msgr2=0x5565244aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:40.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244aecc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c01a040 tx=0x7f2c7c00a580 comp rx=0 tx=0).stop 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 shutdown_connections 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5565245220e0 0x5565245244d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5565244af200 0x5565244ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5565244a6830 0x5565244aecc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 --2- 172.21.15.3:0/3395375546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55652449b650 0x5565244ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.014+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 >> 172.21.15.3:0/3395375546 conn(0x556524497be0 msgr2=0x5565244a4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.016+0000 7f2c7a000640 0 client.25942 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:40.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.016+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 shutdown_connections 2024-08-01T04:20:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.017+0000 7f2c99a3d740 1 -- 172.21.15.3:0/3395375546 wait complete. 2024-08-01T04:20:40.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.018+0000 7f2c99a3d740 20 client.25942 trim_cache size 0 max 16384 2024-08-01T04:20:40.020 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:20:40.020 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:20:40.020 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "8bf1035a-dead-4309-9064-faa4d8ed1d5e", 2024-08-01T04:20:40.020 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:20:40.020 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:20:40.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.020+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2c740a2e40 msgr2=0x7f2c740a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.020+0000 7f2c95a00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2c740a2e40 0x7f2c740a5300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x556524498f60 tx=0x7f2c88005d80 comp rx=0 tx=0).stop 2024-08-01T04:20:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.020+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 msgr2=0x7f2c901b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.020+0000 7f2c95a00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 0x7f2c901b0ce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d950 tx=0x7f2c8000dd50 comp rx=0 tx=0).stop 2024-08-01T04:20:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.021+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 shutdown_connections 2024-08-01T04:20:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.021+0000 7f2c95a00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9010df90 0x7f2c901aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.021+0000 7f2c95a00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2c740a2e40 0x7f2c740a5300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.021+0000 7f2c95a00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c90074c10 0x7f2c901b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.021+0000 7f2c95a00640 1 --2- 172.21.15.3:0/3081076057 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c900742f0 0x7f2c901b0ce0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.021+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 >> 172.21.15.3:0/3081076057 conn(0x7f2c900fd470 msgr2=0x7f2c900fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.022+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 shutdown_connections 2024-08-01T04:20:40.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:40.022+0000 7f2c95a00640 1 -- 172.21.15.3:0/3081076057 wait complete. 2024-08-01T04:20:40.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:40 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:20:40.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:40 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:20:40.302 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:40 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:20:40.322 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:40 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:20:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:40 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:20:40.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:40 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:20:40.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:40 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:20:40.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:40 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:20:40.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:40 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:20:40.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:20:41.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:41 smithi003 ceph-mon[32262]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 319 KiB/s wr, 81 op/s 2024-08-01T04:20:41.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:41 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:20:41.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:41 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:41.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:41 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:20:41.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.630+0000 7fc9c2c00640 1 -- 172.21.15.3:0/2980832708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc105ac0 msgr2=0x7fc9bc105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:41.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.634+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/2980832708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc105ea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009860 tx=0x7fc9ac02a380 comp rx=0 tx=0).stop 2024-08-01T04:20:41.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.634+0000 7fc9c2c00640 1 -- 172.21.15.3:0/2980832708 shutdown_connections 2024-08-01T04:20:41.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.634+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/2980832708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc10daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:41.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.634+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/2980832708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc068fc0 0x7fc9bc100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:41.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.634+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/2980832708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc105ea0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:41.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.634+0000 7fc9c2c00640 1 -- 172.21.15.3:0/2980832708 >> 172.21.15.3:0/2980832708 conn(0x7fc9bc0fb7c0 msgr2=0x7fc9bc0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:41.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.635+0000 7fc9c2c00640 1 -- 172.21.15.3:0/2980832708 shutdown_connections 2024-08-01T04:20:41.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.635+0000 7fc9c2c00640 1 -- 172.21.15.3:0/2980832708 wait complete. 2024-08-01T04:20:41.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.636+0000 7fc9c2c00640 1 Processor -- start 2024-08-01T04:20:41.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.636+0000 7fc9c2c00640 1 -- start start 2024-08-01T04:20:41.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.637+0000 7fc9c2c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc068fc0 0x7fc9bc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:41.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.637+0000 7fc9c2c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:41.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9bb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:41.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9bb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36214/0 (socket says 172.21.15.3:36214) 2024-08-01T04:20:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9bbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc068fc0 0x7fc9bc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9bb400640 1 -- 172.21.15.3:0/361970569 learned_addr learned my addr 172.21.15.3:0/361970569 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc10fab0 con 0x7fc9bc0ffac0 2024-08-01T04:20:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc10f930 con 0x7fc9bc105ac0 2024-08-01T04:20:41.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc10fc30 con 0x7fc9bc068fc0 2024-08-01T04:20:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.638+0000 7fc9c0c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.639+0000 7fc9c0c00640 1 -- 172.21.15.3:0/361970569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc068fc0 msgr2=0x7fc9bc19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.639+0000 7fc9c0c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc068fc0 0x7fc9bc19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.639+0000 7fc9c0c00640 1 -- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 msgr2=0x7fc9bc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.639+0000 7fc9c0c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:41.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.639+0000 7fc9c0c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc19f2b0 con 0x7fc9bc105ac0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.639+0000 7fc9c0c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc19ebd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400ed70 tx=0x7fc9b400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400ee60 con 0x7fc9bc105ac0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9bb400640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc19f5a0 con 0x7fc9bc105ac0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9bbe00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc068fc0 0x7fc9bc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc1101f0 con 0x7fc9bc105ac0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b4003b70 con 0x7fc9bc105ac0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.640+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400ee60 con 0x7fc9bc105ac0 2024-08-01T04:20:41.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.642+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c0052c0 con 0x7fc9bc105ac0 2024-08-01T04:20:41.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.646+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc9b4004a40 con 0x7fc9bc105ac0 2024-08-01T04:20:41.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.648+0000 7fc9b8c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc9a00a30f0 0x7fc9a00a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:41.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.650+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc9b4003970 con 0x7fc9bc105ac0 2024-08-01T04:20:41.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.650+0000 7fc9bbe00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc9a00a30f0 0x7fc9a00a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:41.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.650+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc9b409bd90 con 0x7fc9bc105ac0 2024-08-01T04:20:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:41.654+0000 7fc9bbe00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc9a00a30f0 0x7fc9a00a55b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0095f0 tx=0x7fc9ac005d30 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:41.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:41 smithi163 ceph-mon[37383]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 319 KiB/s wr, 81 op/s 2024-08-01T04:20:41.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:41 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:20:41.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:41 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:41.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:41 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:20:41.883 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:20:41.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:41 smithi160 ceph-mon[37224]: pgmap v135: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 319 KiB/s wr, 81 op/s 2024-08-01T04:20:41.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:41 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:20:41.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:41 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:41.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:41 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:20:42.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.105+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc99c0055b0 con 0x7fc9bc105ac0 2024-08-01T04:20:42.108 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:20:42.108 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:20:42.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.106+0000 7fc9b8c00640 1 -- 172.21.15.3:0/361970569 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7fc9b409b690 con 0x7fc9bc105ac0 2024-08-01T04:20:42.108 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:20:42.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc9a00a30f0 msgr2=0x7fc9a00a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:42.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc9a00a30f0 0x7fc9a00a55b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0095f0 tx=0x7fc9ac005d30 comp rx=0 tx=0).stop 2024-08-01T04:20:42.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc105ac0 msgr2=0x7fc9bc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:42.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc19ebd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400ed70 tx=0x7fc9b400c6e0 comp rx=0 tx=0).stop 2024-08-01T04:20:42.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 shutdown_connections 2024-08-01T04:20:42.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9bc105ac0 0x7fc9bc19ebd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:42.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9bc0ffac0 0x7fc9bc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:42.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc9a00a30f0 0x7fc9a00a55b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:42.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 --2- 172.21.15.3:0/361970569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9bc068fc0 0x7fc9bc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:42.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 >> 172.21.15.3:0/361970569 conn(0x7fc9bc0fb7c0 msgr2=0x7fc9bc10bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:42.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.113+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 shutdown_connections 2024-08-01T04:20:42.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:42.114+0000 7fc9c2c00640 1 -- 172.21.15.3:0/361970569 wait complete. 2024-08-01T04:20:42.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:42 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/361970569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:42.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:42 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/361970569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:42.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:42 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/361970569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "scrub active (131 inodes in the stack)", 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout: "8bf1035a-dead-4309-9064-faa4d8ed1d5e": { 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "8bf1035a-dead-4309-9064-faa4d8ed1d5e", 2024-08-01T04:20:43.426 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:20:43.427 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:43.427 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:20:43.427 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:20:43.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.075+0000 7fe81ea00640 1 -- 172.21.15.3:0/2843858877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 msgr2=0x7fe81810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.075+0000 7fe81ea00640 1 --2- 172.21.15.3:0/2843858877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81810e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe810009ca0 tx=0x7fe81002a8b0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.076+0000 7fe81ea00640 1 -- 172.21.15.3:0/2843858877 shutdown_connections 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.076+0000 7fe81ea00640 1 --2- 172.21.15.3:0/2843858877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81810e770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.076+0000 7fe81ea00640 1 --2- 172.21.15.3:0/2843858877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8181018a0 0x7fe818101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.076+0000 7fe81ea00640 1 --2- 172.21.15.3:0/2843858877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8181078a0 0x7fe818107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.076+0000 7fe81ea00640 1 -- 172.21.15.3:0/2843858877 >> 172.21.15.3:0/2843858877 conn(0x7fe8180fd5c0 msgr2=0x7fe8180ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.077+0000 7fe81ea00640 1 -- 172.21.15.3:0/2843858877 shutdown_connections 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.078+0000 7fe81ea00640 1 -- 172.21.15.3:0/2843858877 wait complete. 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.080+0000 7fe81ea00640 1 Processor -- start 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 -- start start 2024-08-01T04:20:43.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8181018a0 0x7fe81819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8181078a0 0x7fe81819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe818113eb0 con 0x7fe8181018a0 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe818113d30 con 0x7fe818102240 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.081+0000 7fe81ea00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe818114030 con 0x7fe8181078a0 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52300/0 (socket says 172.21.15.3:52300) 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 -- 172.21.15.3:0/1350604725 learned_addr learned my addr 172.21.15.3:0/1350604725 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 -- 172.21.15.3:0/1350604725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8181078a0 msgr2=0x7fe81819ec40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8181078a0 0x7fe81819ec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 -- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8181018a0 msgr2=0x7fe81819a370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-01T04:20:43.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8181018a0 0x7fe81819a370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.082+0000 7fe817400640 1 -- 172.21.15.3:0/1350604725 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81819f340 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.083+0000 7fe817400640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81819a8b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe80800ea60 tx=0x7fe80800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.083+0000 7fe814c00640 1 -- 172.21.15.3:0/1350604725 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80800cde0 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.083+0000 7fe814c00640 1 -- 172.21.15.3:0/1350604725 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8080035f0 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.083+0000 7fe814c00640 1 -- 172.21.15.3:0/1350604725 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80800cde0 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.085+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81819f630 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.086+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81806b940 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.098+0000 7fe814c00640 1 -- 172.21.15.3:0/1350604725 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe8080037a0 con 0x7fe818102240 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.102+0000 7fe8215d9740 1 Processor -- start 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.102+0000 7fe8215d9740 1 -- start start 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.103+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.103+0000 7fe814c00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe7fc0a2fb0 0x7fe7fc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.103+0000 7fe814c00640 1 -- 172.21.15.3:0/1350604725 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe8080cf350 con 0x7fe818102240 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.105+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 0x55f9f25de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.105+0000 7fe817e00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe7fc0a2fb0 0x7fe7fc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe8215d9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f9f2539bf0 con 0x55f9f25d3b90 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe817400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 0x55f9f25de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe817400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 0x55f9f25de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50422/0 (socket says 172.21.15.3:50422) 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe817400640 1 -- 172.21.15.3:0/3831230365 learned_addr learned my addr 172.21.15.3:0/3831230365 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f9f24d1bd0 con 0x55f9f25ded70 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.106+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f9f2535bb0 con 0x55f9f25d44b0 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.107+0000 7fe817e00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe7fc0a2fb0 0x7fe7fc0a5470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b910 tx=0x7fe80400b2f0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.107+0000 7fe81ca00640 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.107+0000 7fe81ca00640 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.107+0000 7fe817400640 1 -- 172.21.15.3:0/3831230365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 msgr2=0x55f9f25e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.108+0000 7fe817400640 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.108+0000 7fe817400640 1 -- 172.21.15.3:0/3831230365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 msgr2=0x55f9f25d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.108+0000 7fe817400640 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25d3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.110+0000 7fe817400640 1 -- 172.21.15.3:0/3831230365 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f24d88a0 con 0x55f9f25d44b0 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.110+0000 7fe81ca00640 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.113+0000 7fe817400640 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 0x55f9f25de830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe808098200 tx=0x7fe8080056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c190e32d1ac0ce server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.113+0000 7fe80d200640 1 -- 172.21.15.3:0/3831230365 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80809b830 con 0x55f9f25d44b0 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.113+0000 7fe80d200640 1 -- 172.21.15.3:0/3831230365 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80809fbe0 con 0x55f9f25d44b0 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.114+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 msgr2=0x55f9f25de830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.114+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 0x55f9f25de830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe808098200 tx=0x7fe8080056d0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.115+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 shutdown_connections 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.115+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.115+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d44b0 0x55f9f25de830 secure :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe808098200 tx=0x7fe8080056d0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.115+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3831230365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25d3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.115+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 >> 172.21.15.3:0/3831230365 conn(0x55f9f25d0120 msgr2=0x55f9f25d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.123+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 shutdown_connections 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.123+0000 7fe8215d9740 1 -- 172.21.15.3:0/3831230365 wait complete. 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.123+0000 7fe8215d9740 1 -- start start 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.124+0000 7fe8215d9740 1 Processor -- start 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.128+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25d3b90 0x55f9f25dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.128+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.128+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.128+0000 7fe8215d9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f9f25e1990 con 0x55f9f25ded70 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.129+0000 7fe81ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.129+0000 7fe81ca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25eae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50432/0 (socket says 172.21.15.3:50432) 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.129+0000 7fe81ca00640 1 -- 172.21.15.3:0/3802646459 learned_addr learned my addr 172.21.15.3:0/3802646459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.129+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f9f24f4d40 con 0x55f9f25d3b90 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.129+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55f9f25e1b10 con 0x55f9f25dca10 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.129+0000 7fe8215d9740 10 client.0 _ll_register_callbacks cb 0x55f9f24bdb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.130+0000 7fe817e00640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25d3b90 0x55f9f25dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.130+0000 7fe817400640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.131+0000 7fe817e00640 1 -- 172.21.15.3:0/3802646459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 msgr2=0x55f9f25eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.131+0000 7fe817e00640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.131+0000 7fe817e00640 1 -- 172.21.15.3:0/3802646459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 msgr2=0x55f9f25dc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.132+0000 7fe817e00640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25dc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.132+0000 7fe817e00640 1 -- 172.21.15.3:0/3802646459 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f22ab3c0 con 0x55f9f25d3b90 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.133+0000 7fe817400640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.134+0000 7fe801600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.134+0000 7fe817e00640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25d3b90 0x55f9f25dbf40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe804005fd0 tx=0x7fe804004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80a56dd057057da0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.135+0000 7fe801600640 1 -- 172.21.15.3:0/3802646459 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804021070 con 0x55f9f25d3b90 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.135+0000 7fe801600640 1 -- 172.21.15.3:0/3802646459 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe80401d5f0 con 0x55f9f25d3b90 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.136+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9f2547160 con 0x55f9f25d3b90 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.136+0000 7fe801600640 1 -- 172.21.15.3:0/3802646459 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe80401d7f0 con 0x55f9f25d3b90 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.137+0000 7fe8215d9740 10 client.25970 fetch_fsmap learned FSMap version 29 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.137+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9f25eddd0 con 0x55f9f25d3b90 2024-08-01T04:20:43.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.137+0000 7fe801600640 1 -- 172.21.15.3:0/3802646459 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f9f2547160 con 0x55f9f25d3b90 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.141+0000 7fe801600640 1 -- 172.21.15.3:0/3802646459 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fe80401c070 con 0x55f9f25d3b90 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.142+0000 7fe8215d9740 10 client.25970 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.142+0000 7fe8215d9740 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.142+0000 7fe8215d9740 20 client.25970 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.142+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f25ef1e0 0x55f9f25f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.142+0000 7fe8215d9740 4 client.25970 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.142+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f2541670 con 0x55f9f25ef1e0 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.146+0000 7fe817400640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f25ef1e0 0x55f9f25f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.148+0000 7fe817400640 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f25ef1e0 0x55f9f25f15d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70d9fb0c2aa3310 server_cookie=9614693553bf2f74 in_seq=0 out_seq=0 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.148+0000 7fe801600640 10 client.25970 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.150+0000 7fe801600640 1 -- 172.21.15.3:0/3802646459 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9f2541670 con 0x55f9f25ef1e0 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.150+0000 7fe801600640 10 client.25970 handle_command_reply: tid=0 2024-08-01T04:20:43.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.150+0000 7fe8215d9740 1 client.25970 shutdown 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.151+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25d3b90 msgr2=0x55f9f25dbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.151+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25d3b90 0x55f9f25dbf40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe804005fd0 tx=0x7fe804004730 comp rx=0 tx=0).stop 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 shutdown_connections 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f25ef1e0 0x55f9f25f15d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3802646459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25d3b90 0x55f9f25dbf40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 >> 172.21.15.3:0/3802646459 conn(0x55f9f25d0120 msgr2=0x55f9f25cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.152+0000 7fe801600640 0 client.25970 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.153+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 shutdown_connections 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.153+0000 7fe8215d9740 1 -- 172.21.15.3:0/3802646459 wait complete. 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.153+0000 7fe8215d9740 20 client.25970 trim_cache size 0 max 16384 2024-08-01T04:20:43.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.214+0000 7fe8215d9740 1 Processor -- start 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.214+0000 7fe8215d9740 1 -- start start 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe8215d9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8181bac40 con 0x55f9f25d3b90 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe8215d9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8181b8510 con 0x55f9f25ded70 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe8215d9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8180725b0 con 0x55f9f25dca10 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.215+0000 7fe81ca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.216+0000 7fe817400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.216+0000 7fe817400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50436/0 (socket says 172.21.15.3:50436) 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.216+0000 7fe817400640 1 -- 172.21.15.3:0/1049958959 learned_addr learned my addr 172.21.15.3:0/1049958959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:43.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.219+0000 7fe81ca00640 1 -- 172.21.15.3:0/1049958959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 msgr2=0x55f9f25cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.219+0000 7fe81ca00640 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.219+0000 7fe81ca00640 1 -- 172.21.15.3:0/1049958959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 msgr2=0x55f9f25eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.219+0000 7fe81ca00640 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.219+0000 7fe81ca00640 1 -- 172.21.15.3:0/1049958959 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f25da490 con 0x55f9f25ded70 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.222+0000 7fe81ca00640 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25f6e30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe810004af0 tx=0x7fe8100303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8789b8c2a75e4cf server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.223+0000 7fe803e00640 1 -- 172.21.15.3:0/1049958959 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810030690 con 0x55f9f25ded70 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.224+0000 7fe803e00640 1 -- 172.21.15.3:0/1049958959 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe810036040 con 0x55f9f25ded70 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.224+0000 7fe803e00640 1 -- 172.21.15.3:0/1049958959 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810004370 con 0x55f9f25ded70 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.226+0000 7fe8215d9740 1 -- 172.21.15.3:0/1049958959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 msgr2=0x55f9f25f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.226+0000 7fe8215d9740 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25f6e30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe810004af0 tx=0x7fe8100303a0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 -- 172.21.15.3:0/1049958959 shutdown_connections 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25ded70 0x55f9f25f6e30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25dca10 0x55f9f25cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 --2- 172.21.15.3:0/1049958959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25d3b90 0x55f9f25eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 -- 172.21.15.3:0/1049958959 >> 172.21.15.3:0/1049958959 conn(0x55f9f25d0120 msgr2=0x55f9f25cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 -- 172.21.15.3:0/1049958959 shutdown_connections 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.227+0000 7fe8215d9740 1 -- 172.21.15.3:0/1049958959 wait complete. 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.228+0000 7fe8215d9740 1 -- start start 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.228+0000 7fe8215d9740 1 Processor -- start 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.231+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d3b90 0x55f9f25e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25dca10 0x55f9f25e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe8215d9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe81ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe81ca00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25d8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48184/0 (socket says 172.21.15.3:48184) 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe81ca00640 1 -- 172.21.15.3:0/3553902215 learned_addr learned my addr 172.21.15.3:0/3553902215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:20:43.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe818072c20 con 0x55f9f25ded70 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8181b7f00 con 0x55f9f25dca10 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe8180732f0 con 0x55f9f25d3b90 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.232+0000 7fe8215d9740 10 client.0 _ll_register_callbacks cb 0x7fe804005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.234+0000 7fe817400640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25dca10 0x55f9f25e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.234+0000 7fe817e00640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d3b90 0x55f9f25e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.235+0000 7fe817400640 1 -- 172.21.15.3:0/3553902215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d3b90 msgr2=0x55f9f25e6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.235+0000 7fe817400640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d3b90 0x55f9f25e6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.235+0000 7fe817400640 1 -- 172.21.15.3:0/3553902215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 msgr2=0x55f9f25d8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.235+0000 7fe817400640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25d8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.236+0000 7fe817400640 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f25d88b0 con 0x55f9f25dca10 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.237+0000 7fe817400640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25dca10 0x55f9f25e7320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe808097510 tx=0x7fe80809b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfdb3cd14582f3c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:20:43.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.237+0000 7fe801600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.237+0000 7fe801600640 1 -- 172.21.15.3:0/3553902215 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808082e60 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.238+0000 7fe801600640 1 -- 172.21.15.3:0/3553902215 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe808099b30 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.238+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe810036040 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.242+0000 7fe801600640 1 -- 172.21.15.3:0/3553902215 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8080a44a0 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.242+0000 7fe8215d9740 10 client.25976 fetch_fsmap learned FSMap version 29 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.242+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f9f2659500 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.242+0000 7fe801600640 1 -- 172.21.15.3:0/3553902215 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe810036040 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe801600640 1 -- 172.21.15.3:0/3553902215 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fe8080ac020 con 0x55f9f25dca10 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe8215d9740 10 client.25976 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe8215d9740 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe8215d9740 20 client.25976 populate_metadata read hostname 'smithi003' 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f26597f0 0x55f9f265bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:20:43.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe8215d9740 4 client.25976 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.243+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9f24be0c0 con 0x55f9f26597f0 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.244+0000 7fe817e00640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f26597f0 0x55f9f265bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe817e00640 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f26597f0 0x55f9f265bbe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5695edb3c0339c2 server_cookie=3725281374404c00 in_seq=0 out_seq=0 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe801600640 10 client.25976 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe801600640 1 -- 172.21.15.3:0/3553902215 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f9f24be0c0 con 0x55f9f26597f0 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe801600640 10 client.25976 handle_command_reply: tid=0 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe8215d9740 1 client.25976 shutdown 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25dca10 msgr2=0x55f9f25e7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25dca10 0x55f9f25e7320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe808097510 tx=0x7fe80809b730 comp rx=0 tx=0).stop 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 shutdown_connections 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.245+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55f9f26597f0 0x55f9f265bbe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9f25ded70 0x55f9f25d8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f9f25dca10 0x55f9f25e7320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe8215d9740 1 --2- 172.21.15.3:0/3553902215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f9f25d3b90 0x55f9f25e6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 >> 172.21.15.3:0/3553902215 conn(0x55f9f25d0120 msgr2=0x55f9f25cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe801600640 0 client.25976 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 shutdown_connections 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.246+0000 7fe8215d9740 1 -- 172.21.15.3:0/3553902215 wait complete. 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.247+0000 7fe8215d9740 20 client.25976 trim_cache size 0 max 16384 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe7fc0a2fb0 msgr2=0x7fe7fc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe7fc0a2fb0 0x7fe7fc0a5470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b910 tx=0x7fe80400b2f0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 msgr2=0x7fe81819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81819a8b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe80800ea60 tx=0x7fe80800ef20 comp rx=0 tx=0).stop 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 shutdown_connections 2024-08-01T04:20:43.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe7fc0a2fb0 0x7fe7fc0a5470 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8181078a0 0x7fe81819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe818102240 0x7fe81819a8b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 --2- 172.21.15.3:0/1350604725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8181018a0 0x7fe81819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:20:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.248+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 >> 172.21.15.3:0/1350604725 conn(0x7fe8180fd5c0 msgr2=0x7fe8180fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:20:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.249+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 shutdown_connections 2024-08-01T04:20:43.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:20:43.251+0000 7fe81ea00640 1 -- 172.21.15.3:0/1350604725 wait complete. 2024-08-01T04:20:43.448 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db / --timeout 90 --expiration 143.55 --await-for 45.0' 2024-08-01T04:20:43.448 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db / --timeout 90 --expiration 143.55 --await-for 45.0 2024-08-01T04:20:43.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:43 smithi163 ceph-mon[37383]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.9 MiB/s wr, 664 op/s 2024-08-01T04:20:43.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:43 smithi003 ceph-mon[32262]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.9 MiB/s wr, 664 op/s 2024-08-01T04:20:43.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:43 smithi160 ceph-mon[37224]: pgmap v136: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.9 MiB/s wr, 664 op/s 2024-08-01T04:20:44.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:44 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:20:44.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:20:44.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:20:44.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:20:44.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:44 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:20:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:20:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:20:44.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:20:44.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:44.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:44 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:20:45.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:45 smithi163 ceph-mon[37383]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 15 MiB/s wr, 3.35k op/s 2024-08-01T04:20:45.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:45 smithi003 ceph-mon[32262]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 15 MiB/s wr, 3.35k op/s 2024-08-01T04:20:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:45 smithi160 ceph-mon[37224]: pgmap v137: 97 pgs: 97 active+clean; 16 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 15 MiB/s wr, 3.35k op/s 2024-08-01T04:20:47.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:47 smithi163 ceph-mon[37383]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 3.89k op/s 2024-08-01T04:20:47.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:47 smithi003 ceph-mon[32262]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 3.89k op/s 2024-08-01T04:20:47.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:47 smithi160 ceph-mon[37224]: pgmap v138: 97 pgs: 97 active+clean; 16 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 3.89k op/s 2024-08-01T04:20:49.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:49 smithi163 ceph-mon[37383]: pgmap v139: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 MiB/s wr, 7.32k op/s 2024-08-01T04:20:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:49 smithi003 ceph-mon[32262]: pgmap v139: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 MiB/s wr, 7.32k op/s 2024-08-01T04:20:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:49 smithi160 ceph-mon[37224]: pgmap v139: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 MiB/s wr, 7.32k op/s 2024-08-01T04:20:51.854 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:51 smithi003 ceph-mon[32262]: pgmap v140: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 39 MiB/s wr, 7.76k op/s 2024-08-01T04:20:51.854 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:51 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:20:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:51 smithi160 ceph-mon[37224]: pgmap v140: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 39 MiB/s wr, 7.76k op/s 2024-08-01T04:20:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:51 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:20:52.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:51 smithi163 ceph-mon[37383]: pgmap v140: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 39 MiB/s wr, 7.76k op/s 2024-08-01T04:20:52.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:51 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:20:52.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:52 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:20:52.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:52 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:20:53.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:52 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:20:53.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:52 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:20:53.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:52 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:20:53.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:52 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:20:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:53 smithi160 ceph-mon[37224]: pgmap v141: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 45 MiB/s wr, 8.68k op/s 2024-08-01T04:20:54.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:53 smithi163 ceph-mon[37383]: pgmap v141: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 45 MiB/s wr, 8.68k op/s 2024-08-01T04:20:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:53 smithi003 ceph-mon[32262]: pgmap v141: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 45 MiB/s wr, 8.68k op/s 2024-08-01T04:20:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:55 smithi160 ceph-mon[37224]: pgmap v142: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 62 MiB/s wr, 11.37k op/s 2024-08-01T04:20:56.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:55 smithi163 ceph-mon[37383]: pgmap v142: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 62 MiB/s wr, 11.37k op/s 2024-08-01T04:20:56.065 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:55 smithi003 ceph-mon[32262]: pgmap v142: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 62 MiB/s wr, 11.37k op/s 2024-08-01T04:20:57.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:57 smithi160 ceph-mon[37224]: pgmap v143: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 56 MiB/s wr, 9.66k op/s 2024-08-01T04:20:58.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:57 smithi163 ceph-mon[37383]: pgmap v143: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 56 MiB/s wr, 9.66k op/s 2024-08-01T04:20:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:57 smithi003 ceph-mon[32262]: pgmap v143: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 56 MiB/s wr, 9.66k op/s 2024-08-01T04:20:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:20:59 smithi160 ceph-mon[37224]: pgmap v144: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 76 MiB/s wr, 12.52k op/s 2024-08-01T04:21:00.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:20:59 smithi163 ceph-mon[37383]: pgmap v144: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 76 MiB/s wr, 12.52k op/s 2024-08-01T04:21:00.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:20:59 smithi003 ceph-mon[32262]: pgmap v144: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 76 MiB/s wr, 12.52k op/s 2024-08-01T04:21:01.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:01 smithi160 ceph-mon[37224]: pgmap v145: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 59 MiB/s wr, 9.64k op/s 2024-08-01T04:21:02.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:01 smithi163 ceph-mon[37383]: pgmap v145: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 59 MiB/s wr, 9.64k op/s 2024-08-01T04:21:02.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:01 smithi003 ceph-mon[32262]: pgmap v145: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 59 MiB/s wr, 9.64k op/s 2024-08-01T04:21:04.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:03 smithi163 ceph-mon[37383]: pgmap v146: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 68 MiB/s wr, 10.88k op/s 2024-08-01T04:21:04.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:03 smithi003 ceph-mon[32262]: pgmap v146: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 68 MiB/s wr, 10.88k op/s 2024-08-01T04:21:04.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:03 smithi160 ceph-mon[37224]: pgmap v146: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 68 MiB/s wr, 10.88k op/s 2024-08-01T04:21:06.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:05 smithi163 ceph-mon[37383]: pgmap v147: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 75 MiB/s wr, 11.77k op/s 2024-08-01T04:21:06.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:05 smithi003 ceph-mon[32262]: pgmap v147: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 75 MiB/s wr, 11.77k op/s 2024-08-01T04:21:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:05 smithi160 ceph-mon[37224]: pgmap v147: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 75 MiB/s wr, 11.77k op/s 2024-08-01T04:21:07.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:07 smithi160 ceph-mon[37224]: pgmap v148: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 59 MiB/s wr, 9.01k op/s 2024-08-01T04:21:08.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:07 smithi163 ceph-mon[37383]: pgmap v148: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 59 MiB/s wr, 9.01k op/s 2024-08-01T04:21:08.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:07 smithi003 ceph-mon[32262]: pgmap v148: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 59 MiB/s wr, 9.01k op/s 2024-08-01T04:21:10.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:09 smithi163 ceph-mon[37383]: pgmap v149: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 82 MiB/s wr, 12.27k op/s 2024-08-01T04:21:10.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:09 smithi003 ceph-mon[32262]: pgmap v149: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 82 MiB/s wr, 12.27k op/s 2024-08-01T04:21:10.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:09 smithi160 ceph-mon[37224]: pgmap v149: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 82 MiB/s wr, 12.27k op/s 2024-08-01T04:21:12.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:11 smithi163 ceph-mon[37383]: pgmap v150: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 67 MiB/s wr, 9.91k op/s 2024-08-01T04:21:12.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:11 smithi003 ceph-mon[32262]: pgmap v150: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 67 MiB/s wr, 9.91k op/s 2024-08-01T04:21:12.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:11 smithi160 ceph-mon[37224]: pgmap v150: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 67 MiB/s wr, 9.91k op/s 2024-08-01T04:21:14.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:13 smithi163 ceph-mon[37383]: pgmap v151: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 67 MiB/s wr, 9.73k op/s 2024-08-01T04:21:14.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:13 smithi003 ceph-mon[32262]: pgmap v151: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 67 MiB/s wr, 9.73k op/s 2024-08-01T04:21:14.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:13 smithi160 ceph-mon[37224]: pgmap v151: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 67 MiB/s wr, 9.73k op/s 2024-08-01T04:21:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:15 smithi163 ceph-mon[37383]: pgmap v152: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 83 MiB/s wr, 11.84k op/s 2024-08-01T04:21:16.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:15 smithi003 ceph-mon[32262]: pgmap v152: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 83 MiB/s wr, 11.84k op/s 2024-08-01T04:21:16.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:15 smithi160 ceph-mon[37224]: pgmap v152: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 83 MiB/s wr, 11.84k op/s 2024-08-01T04:21:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:17 smithi163 ceph-mon[37383]: pgmap v153: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 78 MiB/s wr, 10.98k op/s 2024-08-01T04:21:18.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:17 smithi003 ceph-mon[32262]: pgmap v153: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 78 MiB/s wr, 10.98k op/s 2024-08-01T04:21:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:17 smithi160 ceph-mon[37224]: pgmap v153: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 78 MiB/s wr, 10.98k op/s 2024-08-01T04:21:20.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:19 smithi163 ceph-mon[37383]: pgmap v154: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 104 MiB/s wr, 14.79k op/s 2024-08-01T04:21:20.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:19 smithi003 ceph-mon[32262]: pgmap v154: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 104 MiB/s wr, 14.79k op/s 2024-08-01T04:21:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:19 smithi160 ceph-mon[37224]: pgmap v154: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 104 MiB/s wr, 14.79k op/s 2024-08-01T04:21:21.880 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:21:21.927 INFO:tasks.cephfs.filesystem:scrub status for tag:8bf1035a-dead-4309-9064-faa4d8ed1d5e - {'path': '/', 'tag': '8bf1035a-dead-4309-9064-faa4d8ed1d5e', 'options': 'recursive,force'} 2024-08-01T04:21:22.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:21 smithi163 ceph-mon[37383]: pgmap v155: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 80 MiB/s wr, 11.17k op/s 2024-08-01T04:21:22.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:21 smithi003 ceph-mon[32262]: pgmap v155: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 80 MiB/s wr, 11.17k op/s 2024-08-01T04:21:22.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:21 smithi160 ceph-mon[37224]: pgmap v155: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 80 MiB/s wr, 11.17k op/s 2024-08-01T04:21:23.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.278+0000 7fa4da400640 1 -- 172.21.15.3:0/3799428299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 msgr2=0x7fa4d406e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.278+0000 7fa4da400640 1 --2- 172.21.15.3:0/3799428299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d406e220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009ca0 tx=0x7fa4c402a8c0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.279+0000 7fa4da400640 1 -- 172.21.15.3:0/3799428299 shutdown_connections 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.279+0000 7fa4da400640 1 --2- 172.21.15.3:0/3799428299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d411ae70 0x7fa4d411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.279+0000 7fa4da400640 1 --2- 172.21.15.3:0/3799428299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d406e760 0x7fa4d406eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.279+0000 7fa4da400640 1 --2- 172.21.15.3:0/3799428299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d406e220 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.279+0000 7fa4da400640 1 -- 172.21.15.3:0/3799428299 >> 172.21.15.3:0/3799428299 conn(0x7fa4d4072eb0 msgr2=0x7fa4d40732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.281+0000 7fa4da400640 1 -- 172.21.15.3:0/3799428299 shutdown_connections 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.282+0000 7fa4da400640 1 -- 172.21.15.3:0/3799428299 wait complete. 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.285+0000 7fa4da400640 1 Processor -- start 2024-08-01T04:21:23.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 -- start start 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d406e760 0x7fa4d41a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d41a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d411ae70 0x7fa4d41ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4120910 con 0x7fa4d411ae70 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4120790 con 0x7fa4d406e760 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.286+0000 7fa4da400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d4120a90 con 0x7fa4d410fec0 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d41a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d406e760 0x7fa4d41a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d41a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:36486/0 (socket says 172.21.15.3:36486) 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d3e00640 1 -- 172.21.15.3:0/1314987356 learned_addr learned my addr 172.21.15.3:0/1314987356 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d9000640 1 -- 172.21.15.3:0/1314987356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 msgr2=0x7fa4d41a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:23.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d9000640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d41a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d9000640 1 -- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d411ae70 msgr2=0x7fa4d41ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d9000640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d411ae70 0x7fa4d41ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.287+0000 7fa4d9000640 1 -- 172.21.15.3:0/1314987356 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d41abdc0 con 0x7fa4d406e760 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.289+0000 7fa4d9000640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d406e760 0x7fa4d41a6e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4000f80 tx=0x7fa4c4003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.289+0000 7fa4d1600640 1 -- 172.21.15.3:0/1314987356 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4031650 con 0x7fa4d406e760 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.289+0000 7fa4d1600640 1 -- 172.21.15.3:0/1314987356 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c40317f0 con 0x7fa4d406e760 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.289+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d41ac050 con 0x7fa4d406e760 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.290+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41b3990 con 0x7fa4d406e760 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.290+0000 7fa4d1600640 1 -- 172.21.15.3:0/1314987356 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c402fa30 con 0x7fa4d406e760 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.295+0000 7fa4de65c740 1 Processor -- start 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.295+0000 7fa4de65c740 1 -- start start 2024-08-01T04:21:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 0x55fdda88fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda890360 0x55fdda89a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4d9a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 0x55fdda88fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4d9a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 0x55fdda88fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37302/0 (socket says 172.21.15.3:37302) 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fdda7f4860 con 0x55fdda88fa40 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1954954048 learned_addr learned my addr 172.21.15.3:0/1954954048 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fdda78c0d0 con 0x55fdda89ac20 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.298+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fdda7f0820 con 0x55fdda890360 2024-08-01T04:21:23.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.299+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1954954048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda890360 msgr2=0x55fdda89a6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.299+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda890360 0x55fdda89a6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.299+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1954954048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 msgr2=0x55fdda89d010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.299+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.299+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1954954048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdda792d80 con 0x55fdda88fa40 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.300+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 0x55fdda88fe20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009b50 tx=0x7fa4c000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b26af8b252cc7c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.300+0000 7fa4ca000640 1 -- 172.21.15.3:0/1954954048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0009e40 con 0x55fdda88fa40 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.300+0000 7fa4ca000640 1 -- 172.21.15.3:0/1954954048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c0005500 con 0x55fdda88fa40 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.300+0000 7fa4ca000640 1 -- 172.21.15.3:0/1954954048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0010690 con 0x55fdda88fa40 2024-08-01T04:21:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.301+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 msgr2=0x55fdda88fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:23.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.301+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 0x55fdda88fe20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009b50 tx=0x7fa4c000e720 comp rx=0 tx=0).stop 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.302+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 shutdown_connections 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.302+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.302+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda890360 0x55fdda89a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.302+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1954954048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda88fa40 0x55fdda88fe20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.302+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 >> 172.21.15.3:0/1954954048 conn(0x55fdda88bfd0 msgr2=0x55fdda88c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.303+0000 7fa4d1600640 1 -- 172.21.15.3:0/1314987356 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa4c402fcd0 con 0x7fa4d406e760 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.305+0000 7fa4d1600640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa4b80a2dc0 0x7fa4b80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.306+0000 7fa4d1600640 1 -- 172.21.15.3:0/1314987356 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa4c40f0800 con 0x7fa4d406e760 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.306+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa4b80a2dc0 0x7fa4b80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.307+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa4b80a2dc0 0x7fa4b80a5280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55fdda88d350 tx=0x7fa4cc00b2c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.309+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 shutdown_connections 2024-08-01T04:21:23.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.310+0000 7fa4de65c740 1 -- 172.21.15.3:0/1954954048 wait complete. 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.311+0000 7fa4de65c740 1 -- start start 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.311+0000 7fa4de65c740 1 Processor -- start 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 0x55fdda897c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda89ac20 0x55fdda898150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fdda89d890 con 0x55fdda89ac20 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fdda7af6f0 con 0x55fdda88fa40 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fdda89da10 con 0x55fdda898690 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.314+0000 7fa4de65c740 10 client.0 _ll_register_callbacks cb 0x55fdda79b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.315+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 0x55fdda897c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.315+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 0x55fdda897c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33784/0 (socket says 172.21.15.3:33784) 2024-08-01T04:21:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.315+0000 7fa4d9000640 1 -- 172.21.15.3:0/3730286361 learned_addr learned my addr 172.21.15.3:0/3730286361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.315+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda89ac20 0x55fdda898150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.316+0000 7fa4d9000640 1 -- 172.21.15.3:0/3730286361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 msgr2=0x55fdda8a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.316+0000 7fa4d9000640 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.316+0000 7fa4d9000640 1 -- 172.21.15.3:0/3730286361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda89ac20 msgr2=0x55fdda898150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.316+0000 7fa4d9000640 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda89ac20 0x55fdda898150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.316+0000 7fa4d9000640 1 -- 172.21.15.3:0/3730286361 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdda566ba0 con 0x55fdda88fa40 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.317+0000 7fa4d9000640 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 0x55fdda897c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40ee080 tx=0x7fa4c40b9150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e32937fec1952636 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.317+0000 7fa4bea00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.317+0000 7fa4bea00640 1 -- 172.21.15.3:0/3730286361 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40319a0 con 0x55fdda88fa40 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.317+0000 7fa4bea00640 1 -- 172.21.15.3:0/3730286361 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa4c40c55d0 con 0x55fdda88fa40 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.318+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c0005500 con 0x55fdda88fa40 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.318+0000 7fa4bea00640 1 -- 172.21.15.3:0/3730286361 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4c40da030 con 0x55fdda88fa40 2024-08-01T04:21:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.318+0000 7fa4de65c740 10 client.25985 fetch_fsmap learned FSMap version 29 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.318+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fdda89d890 con 0x55fdda88fa40 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.319+0000 7fa4bea00640 1 -- 172.21.15.3:0/3730286361 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c0005500 con 0x55fdda88fa40 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.319+0000 7fa4bea00640 1 -- 172.21.15.3:0/3730286361 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa4c4004b40 con 0x55fdda88fa40 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.320+0000 7fa4de65c740 10 client.25985 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.320+0000 7fa4de65c740 10 client.25985 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.320+0000 7fa4de65c740 20 client.25985 populate_metadata read hostname 'smithi003' 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.320+0000 7fa4de65c740 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda8aafd0 0x55fdda8ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.320+0000 7fa4de65c740 4 client.25985 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.320+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdda801f90 con 0x55fdda8aafd0 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.321+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda8aafd0 0x55fdda8ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.322+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda8aafd0 0x55fdda8ad470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438ba146f9e53f68 server_cookie=b5bf3f016688c28d in_seq=0 out_seq=0 2024-08-01T04:21:23.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.322+0000 7fa4bea00640 10 client.25985 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.324+0000 7fa4bea00640 1 -- 172.21.15.3:0/3730286361 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdda801f90 con 0x55fdda8aafd0 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.324+0000 7fa4bea00640 10 client.25985 handle_command_reply: tid=0 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 client.25985 shutdown 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 msgr2=0x55fdda897c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 0x55fdda897c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40ee080 tx=0x7fa4c40b9150 comp rx=0 tx=0).stop 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 shutdown_connections 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda8aafd0 0x55fdda8ad470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda89ac20 0x55fdda898150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 --2- 172.21.15.3:0/3730286361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda88fa40 0x55fdda897c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4bea00640 0 client.25985 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.325+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 >> 172.21.15.3:0/3730286361 conn(0x55fdda88bfd0 msgr2=0x55fdda891cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:21:23.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.326+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 shutdown_connections 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.326+0000 7fa4de65c740 1 -- 172.21.15.3:0/3730286361 wait complete. 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.327+0000 7fa4de65c740 20 client.25985 trim_cache size 0 max 16384 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.365+0000 7fa4de65c740 1 Processor -- start 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.365+0000 7fa4de65c740 1 -- start start 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.366+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda898690 0x55fdda8a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.366+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda8b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.366+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 0x55fdda8b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.366+0000 7fa4de65c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4011130 con 0x55fdda898690 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.366+0000 7fa4de65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b400ea00 con 0x55fdda89ac20 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.366+0000 7fa4de65c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b400c850 con 0x55fdda896c10 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.367+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda898690 0x55fdda8a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.367+0000 7fa4d3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda8b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.367+0000 7fa4d9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 0x55fdda8b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.367+0000 7fa4d9a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 0x55fdda8b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:36510/0 (socket says 172.21.15.3:36510) 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.367+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1274441086 learned_addr learned my addr 172.21.15.3:0/1274441086 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.368+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1274441086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 msgr2=0x55fdda8b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.368+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda8b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.368+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1274441086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda898690 msgr2=0x55fdda8a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.368+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda898690 0x55fdda8a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.368+0000 7fa4d9a00640 1 -- 172.21.15.3:0/1274441086 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdda8878c0 con 0x55fdda896c10 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.369+0000 7fa4d9000640 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda898690 0x55fdda8a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.369+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 0x55fdda8b32c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00096d0 tx=0x7fa4c000eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6423e50950957e90 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.370+0000 7fa4c9600640 1 -- 172.21.15.3:0/1274441086 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0009c40 con 0x55fdda896c10 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.370+0000 7fa4c9600640 1 -- 172.21.15.3:0/1274441086 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c0004520 con 0x55fdda896c10 2024-08-01T04:21:23.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.370+0000 7fa4c9600640 1 -- 172.21.15.3:0/1274441086 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00047a0 con 0x55fdda896c10 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 -- 172.21.15.3:0/1274441086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 msgr2=0x55fdda8b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 0x55fdda8b32c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00096d0 tx=0x7fa4c000eed0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 -- 172.21.15.3:0/1274441086 shutdown_connections 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda896c10 0x55fdda8b32c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda8b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 --2- 172.21.15.3:0/1274441086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda898690 0x55fdda8a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.371+0000 7fa4de65c740 1 -- 172.21.15.3:0/1274441086 >> 172.21.15.3:0/1274441086 conn(0x55fdda88bfd0 msgr2=0x55fdda896500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.372+0000 7fa4de65c740 1 -- 172.21.15.3:0/1274441086 shutdown_connections 2024-08-01T04:21:23.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.372+0000 7fa4de65c740 1 -- 172.21.15.3:0/1274441086 wait complete. 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.372+0000 7fa4de65c740 1 -- start start 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.373+0000 7fa4de65c740 1 Processor -- start 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda896c10 0x55fdda8a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b400cec0 con 0x55fdda896c10 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b400e330 con 0x55fdda89ac20 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b400d590 con 0x55fdda898690 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4de65c740 10 client.0 _ll_register_callbacks cb 0x7fa4c40b8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.374+0000 7fa4d3e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda896c10 0x55fdda8a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d9000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda896c10 0x55fdda8a2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37324/0 (socket says 172.21.15.3:37324) 2024-08-01T04:21:23.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d9000640 1 -- 172.21.15.3:0/4020059580 learned_addr learned my addr 172.21.15.3:0/4020059580 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d9a00640 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d3e00640 1 -- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 msgr2=0x55fdda89fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d3e00640 1 -- 172.21.15.3:0/4020059580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda896c10 msgr2=0x55fdda8a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda896c10 0x55fdda8a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.375+0000 7fa4d3e00640 1 -- 172.21.15.3:0/4020059580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdda8a03d0 con 0x55fdda898690 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.376+0000 7fa4d3e00640 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a3410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55fdda899250 tx=0x7fa4cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc54c1760f830240 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.376+0000 7fa4bea00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.376+0000 7fa4bea00640 1 -- 172.21.15.3:0/4020059580 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc004230 con 0x55fdda898690 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.376+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c0004520 con 0x55fdda898690 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.377+0000 7fa4bea00640 1 -- 172.21.15.3:0/4020059580 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa4cc0043d0 con 0x55fdda898690 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.377+0000 7fa4bea00640 1 -- 172.21.15.3:0/4020059580 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4cc0035f0 con 0x55fdda898690 2024-08-01T04:21:23.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.377+0000 7fa4de65c740 10 client.25972 fetch_fsmap learned FSMap version 29 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.377+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fdda916790 con 0x55fdda898690 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.377+0000 7fa4bea00640 1 -- 172.21.15.3:0/4020059580 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c0004520 con 0x55fdda898690 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4bea00640 1 -- 172.21.15.3:0/4020059580 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa4cc02f8e0 con 0x55fdda898690 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4de65c740 10 client.25972 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4de65c740 10 client.25972 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4de65c740 20 client.25972 populate_metadata read hostname 'smithi003' 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4de65c740 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda916fd0 0x55fdda9193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4de65c740 4 client.25972 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.55, "await_for": 45.0}] 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.378+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.55, "await_for": 45.0}) -- 0x55fdda7fc2e0 con 0x55fdda916fd0 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.379+0000 7fa4d9000640 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda916fd0 0x55fdda9193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.380+0000 7fa4d9000640 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda916fd0 0x55fdda9193f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bc5992764927ed server_cookie=33b6d9c6844c3f39 in_seq=0 out_seq=0 2024-08-01T04:21:23.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:23.380+0000 7fa4bea00640 10 client.25972 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:21:24.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:23 smithi003 ceph-mon[32262]: pgmap v156: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 77 MiB/s wr, 10.78k op/s 2024-08-01T04:21:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:23 smithi160 ceph-mon[37224]: pgmap v156: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 77 MiB/s wr, 10.78k op/s 2024-08-01T04:21:24.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:23 smithi163 ceph-mon[37383]: pgmap v156: 97 pgs: 97 active+clean; 19 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 77 MiB/s wr, 10.78k op/s 2024-08-01T04:21:26.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:25 smithi160 ceph-mon[37224]: pgmap v157: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 98 MiB/s wr, 14.07k op/s 2024-08-01T04:21:26.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:25 smithi163 ceph-mon[37383]: pgmap v157: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 98 MiB/s wr, 14.07k op/s 2024-08-01T04:21:26.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:25 smithi003 ceph-mon[32262]: pgmap v157: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 98 MiB/s wr, 14.07k op/s 2024-08-01T04:21:28.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:27 smithi160 ceph-mon[37224]: pgmap v158: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 78 MiB/s wr, 11.46k op/s 2024-08-01T04:21:28.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:27 smithi163 ceph-mon[37383]: pgmap v158: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 78 MiB/s wr, 11.46k op/s 2024-08-01T04:21:28.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:27 smithi003 ceph-mon[32262]: pgmap v158: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 78 MiB/s wr, 11.46k op/s 2024-08-01T04:21:30.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:29 smithi160 ceph-mon[37224]: pgmap v159: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 MiB/s wr, 13.47k op/s 2024-08-01T04:21:30.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:29 smithi163 ceph-mon[37383]: pgmap v159: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 MiB/s wr, 13.47k op/s 2024-08-01T04:21:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:29 smithi003 ceph-mon[32262]: pgmap v159: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 92 MiB/s wr, 13.47k op/s 2024-08-01T04:21:32.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:31 smithi160 ceph-mon[37224]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 88 MiB/s wr, 13.06k op/s 2024-08-01T04:21:32.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:31 smithi163 ceph-mon[37383]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 88 MiB/s wr, 13.06k op/s 2024-08-01T04:21:32.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:31 smithi003 ceph-mon[32262]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 88 MiB/s wr, 13.06k op/s 2024-08-01T04:21:34.102 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:33 smithi003 ceph-mon[32262]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 MiB/s wr, 14.25k op/s 2024-08-01T04:21:34.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:33 smithi160 ceph-mon[37224]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 MiB/s wr, 14.25k op/s 2024-08-01T04:21:34.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:33 smithi163 ceph-mon[37383]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 95 MiB/s wr, 14.25k op/s 2024-08-01T04:21:35.122 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:34 smithi003 ceph-mon[32262]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 127 MiB/s wr, 19.02k op/s 2024-08-01T04:21:35.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:34 smithi160 ceph-mon[37224]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 127 MiB/s wr, 19.02k op/s 2024-08-01T04:21:35.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:34 smithi163 ceph-mon[37383]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 127 MiB/s wr, 19.02k op/s 2024-08-01T04:21:37.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:37 smithi160 ceph-mon[37224]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 MiB/s wr, 16.29k op/s 2024-08-01T04:21:37.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:37 smithi163 ceph-mon[37383]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 MiB/s wr, 16.29k op/s 2024-08-01T04:21:37.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:37 smithi003 ceph-mon[32262]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 MiB/s wr, 16.29k op/s 2024-08-01T04:21:39.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:39 smithi003 ceph-mon[32262]: pgmap v164: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 118 MiB/s wr, 17.64k op/s 2024-08-01T04:21:39.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:39 smithi160 ceph-mon[37224]: pgmap v164: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 118 MiB/s wr, 17.64k op/s 2024-08-01T04:21:39.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:39 smithi163 ceph-mon[37383]: pgmap v164: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 118 MiB/s wr, 17.64k op/s 2024-08-01T04:21:40.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:40 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:21:40.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:40 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:21:40.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:40 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:21:41.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:41 smithi160 ceph-mon[37224]: pgmap v165: 97 pgs: 97 active+clean; 21 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 126 MiB/s wr, 19.03k op/s 2024-08-01T04:21:41.514 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:41 smithi003 ceph-mon[32262]: pgmap v165: 97 pgs: 97 active+clean; 21 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 126 MiB/s wr, 19.03k op/s 2024-08-01T04:21:41.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:41 smithi163 ceph-mon[37383]: pgmap v165: 97 pgs: 97 active+clean; 21 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 126 MiB/s wr, 19.03k op/s 2024-08-01T04:21:43.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:43 smithi160 ceph-mon[37224]: pgmap v166: 97 pgs: 97 active+clean; 21 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 99 MiB/s wr, 15.11k op/s 2024-08-01T04:21:43.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:43 smithi163 ceph-mon[37383]: pgmap v166: 97 pgs: 97 active+clean; 21 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 99 MiB/s wr, 15.11k op/s 2024-08-01T04:21:43.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:43 smithi003 ceph-mon[32262]: pgmap v166: 97 pgs: 97 active+clean; 21 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 99 MiB/s wr, 15.11k op/s 2024-08-01T04:21:45.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:45 smithi160 ceph-mon[37224]: pgmap v167: 97 pgs: 97 active+clean; 21 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 112 MiB/s wr, 17.08k op/s 2024-08-01T04:21:45.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:45 smithi163 ceph-mon[37383]: pgmap v167: 97 pgs: 97 active+clean; 21 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 112 MiB/s wr, 17.08k op/s 2024-08-01T04:21:45.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:45 smithi003 ceph-mon[32262]: pgmap v167: 97 pgs: 97 active+clean; 21 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 112 MiB/s wr, 17.08k op/s 2024-08-01T04:21:47.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:47 smithi163 ceph-mon[37383]: pgmap v168: 97 pgs: 97 active+clean; 21 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 81 MiB/s wr, 12.28k op/s 2024-08-01T04:21:47.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:47 smithi003 ceph-mon[32262]: pgmap v168: 97 pgs: 97 active+clean; 21 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 81 MiB/s wr, 12.28k op/s 2024-08-01T04:21:47.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:47 smithi160 ceph-mon[37224]: pgmap v168: 97 pgs: 97 active+clean; 21 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 81 MiB/s wr, 12.28k op/s 2024-08-01T04:21:49.762 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:49 smithi003 ceph-mon[32262]: pgmap v169: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 MiB/s wr, 12.76k op/s 2024-08-01T04:21:49.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:49 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:21:49.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:49 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:21:49.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:49 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:21:49.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:49 smithi163 ceph-mon[37383]: pgmap v169: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 MiB/s wr, 12.76k op/s 2024-08-01T04:21:49.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:49 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:21:49.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:49 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:21:49.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:49 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:21:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:49 smithi160 ceph-mon[37224]: pgmap v169: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 MiB/s wr, 12.76k op/s 2024-08-01T04:21:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:49 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:21:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:49 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:21:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:49 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:21:51.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:51 smithi163 ceph-mon[37383]: pgmap v170: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 MiB/s wr, 12.56k op/s 2024-08-01T04:21:51.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:51 smithi003 ceph-mon[32262]: pgmap v170: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 MiB/s wr, 12.56k op/s 2024-08-01T04:21:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:51 smithi160 ceph-mon[37224]: pgmap v170: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 MiB/s wr, 12.56k op/s 2024-08-01T04:21:51.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:21:53.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:53 smithi163 ceph-mon[37383]: pgmap v171: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 MiB/s wr, 8.15k op/s 2024-08-01T04:21:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:53 smithi003 ceph-mon[32262]: pgmap v171: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 MiB/s wr, 8.15k op/s 2024-08-01T04:21:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:53 smithi160 ceph-mon[37224]: pgmap v171: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 MiB/s wr, 8.15k op/s 2024-08-01T04:21:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4bea00640 1 -- 172.21.15.3:0/4020059580 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fdda7fc2e0 con 0x55fdda916fd0 2024-08-01T04:21:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4bea00640 10 client.25972 handle_command_reply: tid=0 2024-08-01T04:21:54.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 client.25972 shutdown 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 msgr2=0x55fdda8a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a3410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55fdda899250 tx=0x7fa4cc004730 comp rx=0 tx=0).stop 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 shutdown_connections 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55fdda916fd0 0x55fdda9193f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fdda89ac20 0x55fdda89fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fdda898690 0x55fdda8a3410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 --2- 172.21.15.3:0/4020059580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fdda896c10 0x55fdda8a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.457+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 >> 172.21.15.3:0/4020059580 conn(0x55fdda88bfd0 msgr2=0x55fdda891490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.458+0000 7fa4bea00640 0 client.25972 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.458+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 shutdown_connections 2024-08-01T04:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.458+0000 7fa4de65c740 1 -- 172.21.15.3:0/4020059580 wait complete. 2024-08-01T04:21:54.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.458+0000 7fa4de65c740 20 client.25972 trim_cache size 0 max 16384 2024-08-01T04:21:54.461 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 29, 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 14, 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "b970358f": { 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 14, 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:21:54.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 143.6, 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:21:54.463 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:21:54.464 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:21:54.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa4b80a2dc0 msgr2=0x7fa4b80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:54.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa4b80a2dc0 0x7fa4b80a5280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55fdda88d350 tx=0x7fa4cc00b2c0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d406e760 msgr2=0x7fa4d41a6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:21:54.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d406e760 0x7fa4d41a6e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4000f80 tx=0x7fa4c4003f10 comp rx=0 tx=0).stop 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 shutdown_connections 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa4d411ae70 0x7fa4d41ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa4b80a2dc0 0x7fa4b80a5280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4d410fec0 0x7fa4d41a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 --2- 172.21.15.3:0/1314987356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d406e760 0x7fa4d41a6e10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.459+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 >> 172.21.15.3:0/1314987356 conn(0x7fa4d4072eb0 msgr2=0x7fa4d411a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.460+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 shutdown_connections 2024-08-01T04:21:54.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:21:54.460+0000 7fa4da400640 1 -- 172.21.15.3:0/1314987356 wait complete. 2024-08-01T04:21:55.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:55 smithi163 ceph-mon[37383]: pgmap v172: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 MiB/s wr, 9.53k op/s 2024-08-01T04:21:55.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:55 smithi003 ceph-mon[32262]: pgmap v172: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 MiB/s wr, 9.53k op/s 2024-08-01T04:21:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:55 smithi160 ceph-mon[37224]: pgmap v172: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 MiB/s wr, 9.53k op/s 2024-08-01T04:21:57.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:57 smithi163 ceph-mon[37383]: pgmap v173: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 46 MiB/s wr, 6.95k op/s 2024-08-01T04:21:57.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:57 smithi003 ceph-mon[32262]: pgmap v173: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 46 MiB/s wr, 6.95k op/s 2024-08-01T04:21:57.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:57 smithi160 ceph-mon[37224]: pgmap v173: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 46 MiB/s wr, 6.95k op/s 2024-08-01T04:21:59.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:21:59 smithi163 ceph-mon[37383]: pgmap v174: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 42 MiB/s wr, 6.33k op/s 2024-08-01T04:21:59.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:21:59 smithi003 ceph-mon[32262]: pgmap v174: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 42 MiB/s wr, 6.33k op/s 2024-08-01T04:21:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:21:59 smithi160 ceph-mon[37224]: pgmap v174: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 42 MiB/s wr, 6.33k op/s 2024-08-01T04:22:01.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:01 smithi163 ceph-mon[37383]: pgmap v175: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 34 MiB/s wr, 5.28k op/s 2024-08-01T04:22:01.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:01 smithi003 ceph-mon[32262]: pgmap v175: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 34 MiB/s wr, 5.28k op/s 2024-08-01T04:22:01.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:01 smithi160 ceph-mon[37224]: pgmap v175: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 34 MiB/s wr, 5.28k op/s 2024-08-01T04:22:03.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:03 smithi163 ceph-mon[37383]: pgmap v176: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 2.94k op/s 2024-08-01T04:22:03.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:03 smithi003 ceph-mon[32262]: pgmap v176: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 2.94k op/s 2024-08-01T04:22:03.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:03 smithi160 ceph-mon[37224]: pgmap v176: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 2.94k op/s 2024-08-01T04:22:05.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:05 smithi163 ceph-mon[37383]: pgmap v177: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 2.96k op/s 2024-08-01T04:22:05.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:05 smithi003 ceph-mon[32262]: pgmap v177: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 2.96k op/s 2024-08-01T04:22:05.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:05 smithi160 ceph-mon[37224]: pgmap v177: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 19 MiB/s wr, 2.96k op/s 2024-08-01T04:22:07.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b970358f', quiesce took 88.1 seconds. Will release after: -64.55 2024-08-01T04:22:07.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:22:07.284 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:22:07.684 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:07 smithi003 ceph-mon[32262]: pgmap v178: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.58k op/s 2024-08-01T04:22:07.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:07 smithi163 ceph-mon[37383]: pgmap v178: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.58k op/s 2024-08-01T04:22:07.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:07 smithi160 ceph-mon[37224]: pgmap v178: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.58k op/s 2024-08-01T04:22:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 -- 172.21.15.3:0/3842356229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 msgr2=0x7f204810e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 --2- 172.21.15.3:0/3842356229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204810e5d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2044009cd0 tx=0x7f204402a8f0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 -- 172.21.15.3:0/3842356229 shutdown_connections 2024-08-01T04:22:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 --2- 172.21.15.3:0/3842356229 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204810e5d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 --2- 172.21.15.3:0/3842356229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 0x7f2048101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 --2- 172.21.15.3:0/3842356229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048107700 0x7f2048107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 -- 172.21.15.3:0/3842356229 >> 172.21.15.3:0/3842356229 conn(0x7f20480fd440 msgr2=0x7f20480ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 -- 172.21.15.3:0/3842356229 shutdown_connections 2024-08-01T04:22:07.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.949+0000 7f204f200640 1 -- 172.21.15.3:0/3842356229 wait complete. 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.950+0000 7f204f200640 1 Processor -- start 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.950+0000 7f204f200640 1 -- start start 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.951+0000 7f204f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 0x7f204819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.951+0000 7f204f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.951+0000 7f204f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048107700 0x7f204819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.951+0000 7f204f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2048111b50 con 0x7f2048101700 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.951+0000 7f204f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f20481119d0 con 0x7f2048107700 2024-08-01T04:22:07.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.951+0000 7f204f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2048111cd0 con 0x7f20481020a0 2024-08-01T04:22:07.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 0x7f204819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 0x7f204819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55172/0 (socket says 172.21.15.3:55172) 2024-08-01T04:22:07.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204de00640 1 -- 172.21.15.3:0/814361423 learned_addr learned my addr 172.21.15.3:0/814361423 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:07.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40546/0 (socket says 172.21.15.3:40546) 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204e800640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048107700 0x7f204819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 -- 172.21.15.3:0/814361423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048107700 msgr2=0x7f204819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048107700 0x7f204819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 -- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 msgr2=0x7f204819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 0x7f204819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 -- 172.21.15.3:0/814361423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204819f1e0 con 0x7f20481020a0 2024-08-01T04:22:07.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.952+0000 7f204d400640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204819a770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f203800e9f0 tx=0x7f203800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:07.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.953+0000 7f2042a00640 1 -- 172.21.15.3:0/814361423 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203800cd80 con 0x7f20481020a0 2024-08-01T04:22:07.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.953+0000 7f2042a00640 1 -- 172.21.15.3:0/814361423 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20380035f0 con 0x7f20481020a0 2024-08-01T04:22:07.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.953+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204819f4d0 con 0x7f20481020a0 2024-08-01T04:22:07.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.953+0000 7f2042a00640 1 -- 172.21.15.3:0/814361423 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038010690 con 0x7f20481020a0 2024-08-01T04:22:07.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.953+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481a6db0 con 0x7f20481020a0 2024-08-01T04:22:07.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.956+0000 7f2053256740 1 Processor -- start 2024-08-01T04:22:07.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 -- start start 2024-08-01T04:22:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2042a00640 1 -- 172.21.15.3:0/814361423 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2038021430 con 0x7f20481020a0 2024-08-01T04:22:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x56410905e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109068a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x56410906b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564108fc3bf0 con 0x56410905dd80 2024-08-01T04:22:07.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x564108f5bbd0 con 0x564109068f60 2024-08-01T04:22:07.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.957+0000 7f2053256740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x564108fbfbb0 con 0x56410905e6a0 2024-08-01T04:22:07.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109068a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x56410905e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109068a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40548/0 (socket says 172.21.15.3:40548) 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 -- 172.21.15.3:0/3444471534 learned_addr learned my addr 172.21.15.3:0/3444471534 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204d400640 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x56410906b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 -- 172.21.15.3:0/3444471534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 msgr2=0x56410906b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x56410906b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 -- 172.21.15.3:0/3444471534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 msgr2=0x56410905e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x56410905e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.958+0000 7f204de00640 1 -- 172.21.15.3:0/3444471534 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564108f628a0 con 0x56410905e6a0 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.959+0000 7f204de00640 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109068a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f203c009da0 tx=0x7f203c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=175c900e1e7927a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.959+0000 7f2037000640 1 -- 172.21.15.3:0/3444471534 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c01e070 con 0x56410905e6a0 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.959+0000 7f2037000640 1 -- 172.21.15.3:0/3444471534 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f203c019b10 con 0x56410905e6a0 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.959+0000 7f2037000640 1 -- 172.21.15.3:0/3444471534 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00fb80 con 0x56410905e6a0 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.960+0000 7f2042a00640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f20300a2d70 0x7f20300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.960+0000 7f2042a00640 1 -- 172.21.15.3:0/814361423 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f20380cf600 con 0x7f20481020a0 2024-08-01T04:22:07.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.960+0000 7f204e800640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f20300a2d70 0x7f20300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.960+0000 7f2053256740 1 -- 172.21.15.3:0/3444471534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 msgr2=0x564109068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:07.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.960+0000 7f2053256740 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109068a20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f203c009da0 tx=0x7f203c011810 comp rx=0 tx=0).stop 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 -- 172.21.15.3:0/3444471534 shutdown_connections 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x56410906b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109068a20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 --2- 172.21.15.3:0/3444471534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x56410905e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 -- 172.21.15.3:0/3444471534 >> 172.21.15.3:0/3444471534 conn(0x56410905a310 msgr2=0x56410905a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 -- 172.21.15.3:0/3444471534 shutdown_connections 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f2053256740 1 -- 172.21.15.3:0/3444471534 wait complete. 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.961+0000 7f204e800640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f20300a2d70 0x7f20300a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2044009430 tx=0x7f20440098a0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 -- start start 2024-08-01T04:22:07.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 Processor -- start 2024-08-01T04:22:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x564109065d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109066240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x564109066780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56410906bb30 con 0x56410905dd80 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x564108f7ed40 con 0x564109068f60 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x56410906bcb0 con 0x56410905e6a0 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.962+0000 7f2053256740 10 client.0 _ll_register_callbacks cb 0x564108f47b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x564109065d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109066240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40556/0 (socket says 172.21.15.3:40556) 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 -- 172.21.15.3:0/4188566126 learned_addr learned my addr 172.21.15.3:0/4188566126 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204e800640 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x564109066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 -- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 msgr2=0x564109066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:07.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x564109066780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 -- 172.21.15.3:0/4188566126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 msgr2=0x564109065d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x564109065d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.963+0000 7f204d400640 1 -- 172.21.15.3:0/4188566126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564108d353c0 con 0x56410905e6a0 2024-08-01T04:22:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f204d400640 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109066240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f203800c6a0 tx=0x7f203809c180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b402e7fb895920 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:07.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f202b400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:22:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f202b400640 1 -- 172.21.15.3:0/4188566126 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380a46f0 con 0x56410905e6a0 2024-08-01T04:22:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f202b400640 1 -- 172.21.15.3:0/4188566126 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f20380980f0 con 0x56410905e6a0 2024-08-01T04:22:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203c019b10 con 0x56410905e6a0 2024-08-01T04:22:07.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f202b400640 1 -- 172.21.15.3:0/4188566126 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f20380ccf30 con 0x56410905e6a0 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f2053256740 10 client.25984 fetch_fsmap learned FSMap version 29 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.964+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56410907fff0 con 0x56410905e6a0 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f202b400640 1 -- 172.21.15.3:0/4188566126 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f203c019b10 con 0x56410905e6a0 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f202b400640 1 -- 172.21.15.3:0/4188566126 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f20380cf1a0 con 0x56410905e6a0 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f2053256740 10 client.25984 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f2053256740 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f2053256740 20 client.25984 populate_metadata read hostname 'smithi003' 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f2053256740 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090814b0 0x5641090838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f2053256740 4 client.25984 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564108fd1160 con 0x5641090814b0 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.965+0000 7f204de00640 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090814b0 0x5641090838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:07.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.966+0000 7f204de00640 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090814b0 0x5641090838d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e71641bb30152ea server_cookie=d5007cab51f3370d in_seq=0 out_seq=0 2024-08-01T04:22:07.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.966+0000 7f202b400640 10 client.25984 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f202b400640 1 -- 172.21.15.3:0/4188566126 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564108fd1160 con 0x5641090814b0 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f202b400640 10 client.25984 handle_command_reply: tid=0 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 client.25984 shutdown 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 msgr2=0x564109066240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109066240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f203800c6a0 tx=0x7f203809c180 comp rx=0 tx=0).stop 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 shutdown_connections 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090814b0 0x5641090838d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x564109068f60 0x564109066780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109066240 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 --2- 172.21.15.3:0/4188566126 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x564109065d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.967+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 >> 172.21.15.3:0/4188566126 conn(0x56410905a310 msgr2=0x564109060040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.968+0000 7f202b400640 0 client.25984 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.968+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 shutdown_connections 2024-08-01T04:22:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.968+0000 7f2053256740 1 -- 172.21.15.3:0/4188566126 wait complete. 2024-08-01T04:22:07.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:07.968+0000 7f2053256740 20 client.25984 trim_cache size 0 max 16384 2024-08-01T04:22:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.004+0000 7f2053256740 1 Processor -- start 2024-08-01T04:22:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.004+0000 7f2053256740 1 -- start start 2024-08-01T04:22:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f2053256740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x5641090891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f2053256740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905e6a0 0x5641090896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f2053256740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564109068f60 0x56410906a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f2053256740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f202c011180 con 0x56410905dd80 2024-08-01T04:22:08.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f2053256740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f202c00ea50 con 0x56410905e6a0 2024-08-01T04:22:08.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f2053256740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f202c00c8a0 con 0x564109068f60 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x5641090891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204de00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x5641090891a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55194/0 (socket says 172.21.15.3:55194) 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204de00640 1 -- 172.21.15.3:0/4041982752 learned_addr learned my addr 172.21.15.3:0/4041982752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204e800640 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564109068f60 0x56410906a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204d400640 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905e6a0 0x5641090896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204de00640 1 -- 172.21.15.3:0/4041982752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564109068f60 msgr2=0x56410906a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.005+0000 7f204de00640 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564109068f60 0x56410906a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f204de00640 1 -- 172.21.15.3:0/4041982752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905e6a0 msgr2=0x5641090896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f204de00640 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905e6a0 0x5641090896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f204de00640 1 -- 172.21.15.3:0/4041982752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56410905e4d0 con 0x56410905dd80 2024-08-01T04:22:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f204de00640 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x5641090891a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f203c013010 tx=0x7f203c01aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe05701e362924c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f2036600640 1 -- 172.21.15.3:0/4041982752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c01e040 con 0x56410905dd80 2024-08-01T04:22:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f2036600640 1 -- 172.21.15.3:0/4041982752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f203c01f040 con 0x56410905dd80 2024-08-01T04:22:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.006+0000 7f2036600640 1 -- 172.21.15.3:0/4041982752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c004680 con 0x56410905dd80 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.007+0000 7f2053256740 1 -- 172.21.15.3:0/4041982752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 msgr2=0x5641090891a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.007+0000 7f2053256740 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x5641090891a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f203c013010 tx=0x7f203c01aee0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 -- 172.21.15.3:0/4041982752 shutdown_connections 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564109068f60 0x56410906a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905e6a0 0x5641090896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 --2- 172.21.15.3:0/4041982752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56410905dd80 0x5641090891a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 -- 172.21.15.3:0/4041982752 >> 172.21.15.3:0/4041982752 conn(0x56410905a310 msgr2=0x564109060040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:08.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 -- 172.21.15.3:0/4041982752 shutdown_connections 2024-08-01T04:22:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 -- 172.21.15.3:0/4041982752 wait complete. 2024-08-01T04:22:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.008+0000 7f2053256740 1 -- start start 2024-08-01T04:22:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 Processor -- start 2024-08-01T04:22:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905dd80 0x564109069720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109069c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564109068f60 0x564109078820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f202c00cf10 con 0x564109068f60 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f202c00e380 con 0x56410905dd80 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f202c00d5e0 con 0x56410905e6a0 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.009+0000 7f2053256740 10 client.0 _ll_register_callbacks cb 0x7f203809b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40580/0 (socket says 172.21.15.3:40580) 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 -- 172.21.15.3:0/3133540601 learned_addr learned my addr 172.21.15.3:0/3133540601 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204de00640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905dd80 0x564109069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204e800640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564109068f60 0x564109078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 -- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905dd80 msgr2=0x564109069720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905dd80 0x564109069720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 -- 172.21.15.3:0/3133540601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564109068f60 msgr2=0x564109078820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564109068f60 0x564109078820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.010+0000 7f204d400640 1 -- 172.21.15.3:0/3133540601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564109078d60 con 0x56410905e6a0 2024-08-01T04:22:08.012 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.011+0000 7f204d400640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109069c60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f203809ba10 tx=0x7f203809b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bc1a8ebbf6c06c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.011+0000 7f202b400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.011+0000 7f202b400640 1 -- 172.21.15.3:0/3133540601 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380a46f0 con 0x56410905e6a0 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.011+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203c01f040 con 0x56410905e6a0 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.011+0000 7f202b400640 1 -- 172.21.15.3:0/3133540601 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f20380cf110 con 0x56410905e6a0 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f202b400640 1 -- 172.21.15.3:0/3133540601 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f203809b850 con 0x56410905e6a0 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 10 client.25990 fetch_fsmap learned FSMap version 29 2024-08-01T04:22:08.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56410907a070 con 0x56410905e6a0 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f202b400640 1 -- 172.21.15.3:0/3133540601 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f203c01f040 con 0x56410905e6a0 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f202b400640 1 -- 172.21.15.3:0/3133540601 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2038080050 con 0x56410905e6a0 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 10 client.25990 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 20 client.25990 populate_metadata read hostname 'smithi003' 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090ec170 0x5641090ee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 4 client.25990 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.012+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564108fcb670 con 0x5641090ec170 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.013+0000 7f204de00640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090ec170 0x5641090ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.013+0000 7f204de00640 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090ec170 0x5641090ee590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad0e284fc4bb142e server_cookie=144dcb75ad9a46f1 in_seq=0 out_seq=0 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.013+0000 7f202b400640 10 client.25990 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:08.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.013+0000 7f202b400640 1 -- 172.21.15.3:0/3133540601 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564108fcb670 con 0x5641090ec170 2024-08-01T04:22:08.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.013+0000 7f202b400640 10 client.25990 handle_command_reply: tid=0 2024-08-01T04:22:08.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.013+0000 7f2053256740 1 client.25990 shutdown 2024-08-01T04:22:08.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 msgr2=0x564109069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109069c60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f203809ba10 tx=0x7f203809b670 comp rx=0 tx=0).stop 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 shutdown_connections 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5641090ec170 0x5641090ee590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564109068f60 0x564109078820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56410905e6a0 0x564109069c60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 --2- 172.21.15.3:0/3133540601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56410905dd80 0x564109069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f202b400640 0 client.25990 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 >> 172.21.15.3:0/3133540601 conn(0x56410905a310 msgr2=0x5641090645d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 shutdown_connections 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 1 -- 172.21.15.3:0/3133540601 wait complete. 2024-08-01T04:22:08.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.014+0000 7f2053256740 20 client.25990 trim_cache size 0 max 16384 2024-08-01T04:22:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:22:08.018 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:22:08.018 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:22:08.018 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f20300a2d70 msgr2=0x7f20300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f20300a2d70 0x7f20300a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2044009430 tx=0x7f20440098a0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 msgr2=0x7f204819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204819a770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f203800e9f0 tx=0x7f203800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 shutdown_connections 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2048107700 0x7f204819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f20300a2d70 0x7f20300a5230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20481020a0 0x7f204819a770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 --2- 172.21.15.3:0/814361423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2048101700 0x7f204819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.015+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 >> 172.21.15.3:0/814361423 conn(0x7f20480fd440 msgr2=0x7f204810cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.016+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 shutdown_connections 2024-08-01T04:22:08.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.016+0000 7f204f200640 1 -- 172.21.15.3:0/814361423 wait complete. 2024-08-01T04:22:09.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:22:09.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 -- 172.21.15.3:0/2162477868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 msgr2=0x7f9520108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 --2- 172.21.15.3:0/2162477868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f9520108170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f950c009930 tx=0x7f950c02a440 comp rx=0 tx=0).stop 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 -- 172.21.15.3:0/2162477868 shutdown_connections 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 --2- 172.21.15.3:0/2162477868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 0x7f9520114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 --2- 172.21.15.3:0/2162477868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f9520108170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 --2- 172.21.15.3:0/2162477868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f952010dd90 0x7f952010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 -- 172.21.15.3:0/2162477868 >> 172.21.15.3:0/2162477868 conn(0x7f95200776b0 msgr2=0x7f9520079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 -- 172.21.15.3:0/2162477868 shutdown_connections 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.955+0000 7f9525600640 1 -- 172.21.15.3:0/2162477868 wait complete. 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.956+0000 7f9525600640 1 Processor -- start 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.956+0000 7f9525600640 1 -- start start 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.956+0000 7f9525600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f952019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.956+0000 7f9525600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 0x7f952019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.956+0000 7f9525600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f952010dd90 0x7f95201a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f9525600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f95201180e0 con 0x7f9520107d50 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f9525600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9520117f60 con 0x7f952010dd90 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f9525600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9520118260 con 0x7f95201086b0 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f952019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 0x7f952019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:09.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 0x7f952019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40590/0 (socket says 172.21.15.3:40590) 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951f400640 1 -- 172.21.15.3:0/1312743930 learned_addr learned my addr 172.21.15.3:0/1312743930 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f9524c00640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f952010dd90 0x7f95201a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951fe00640 1 -- 172.21.15.3:0/1312743930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 msgr2=0x7f952019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951fe00640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 0x7f952019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951fe00640 1 -- 172.21.15.3:0/1312743930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f952010dd90 msgr2=0x7f95201a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.957+0000 7f951fe00640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f952010dd90 0x7f95201a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f951fe00640 1 -- 172.21.15.3:0/1312743930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95201a3570 con 0x7f9520107d50 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f951fe00640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f952019e5c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f951000e9f0 tx=0x7f951000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000cd80 con 0x7f9520107d50 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95100035f0 con 0x7f9520107d50 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510010690 con 0x7f9520107d50 2024-08-01T04:22:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95201a3860 con 0x7f9520107d50 2024-08-01T04:22:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.958+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201ab140 con 0x7f9520107d50 2024-08-01T04:22:09.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:22:09.366 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:22:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.960+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520108170 con 0x7f9520107d50 2024-08-01T04:22:09.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.961+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9510003b30 con 0x7f9520107d50 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.963+0000 7f951cc00640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95040a2d70 0x7f95040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.963+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f95100cf6d0 con 0x7f9520107d50 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.963+0000 7f951f400640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95040a2d70 0x7f95040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.964+0000 7f951f400640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95040a2d70 0x7f95040a5230 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f950c005c40 tx=0x7f950c005b50 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:08.967+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f951009c250 con 0x7f9520107d50 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.192+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9520110280 con 0x7f9520107d50 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.194+0000 7f951cc00640 1 -- 172.21.15.3:0/1312743930 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f951009bb50 con 0x7f9520107d50 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95040a2d70 msgr2=0x7f95040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95040a2d70 0x7f95040a5230 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f950c005c40 tx=0x7f950c005b50 comp rx=0 tx=0).stop 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 msgr2=0x7f952019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f952019e5c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f951000e9f0 tx=0x7f951000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 shutdown_connections 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f952010dd90 0x7f95201a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.196+0000 7f9525600640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f95040a2d70 0x7f95040a5230 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.197+0000 7f9525600640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95201086b0 0x7f952019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.197+0000 7f9525600640 1 --2- 172.21.15.3:0/1312743930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9520107d50 0x7f952019e5c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.197+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 >> 172.21.15.3:0/1312743930 conn(0x7f95200776b0 msgr2=0x7f9520078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.197+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 shutdown_connections 2024-08-01T04:22:09.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:09.197+0000 7f9525600640 1 -- 172.21.15.3:0/1312743930 wait complete. 2024-08-01T04:22:09.766 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:09 smithi003 ceph-mon[32262]: pgmap v179: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 373 op/s 2024-08-01T04:22:09.766 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:09 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1312743930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:22:09.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:09 smithi163 ceph-mon[37383]: pgmap v179: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 373 op/s 2024-08-01T04:22:09.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:09 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1312743930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:22:09.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:09 smithi160 ceph-mon[37224]: pgmap v179: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 373 op/s 2024-08-01T04:22:09.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:09 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1312743930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:22:10.178 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db --set-id b970358f --release --await' 2024-08-01T04:22:10.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db --set-id b970358f --release --await 2024-08-01T04:22:10.227 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:22:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.871+0000 7f24b4600640 1 -- 172.21.15.3:0/2883637637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 msgr2=0x7f24ac110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.871+0000 7f24b4600640 1 --2- 172.21.15.3:0/2883637637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 0x7f24ac110990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009cd0 tx=0x7f24a402a8e0 comp rx=0 tx=0).stop 2024-08-01T04:22:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 -- 172.21.15.3:0/2883637637 shutdown_connections 2024-08-01T04:22:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 --2- 172.21.15.3:0/2883637637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 0x7f24ac110990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:10.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 --2- 172.21.15.3:0/2883637637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 --2- 172.21.15.3:0/2883637637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ac109ac0 0x7f24ac109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 -- 172.21.15.3:0/2883637637 >> 172.21.15.3:0/2883637637 conn(0x7f24ac0ff7c0 msgr2=0x7f24ac101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 -- 172.21.15.3:0/2883637637 shutdown_connections 2024-08-01T04:22:10.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.872+0000 7f24b4600640 1 -- 172.21.15.3:0/2883637637 wait complete. 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 Processor -- start 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 -- start start 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 0x7f24ac19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ac109ac0 0x7f24ac19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac113f10 con 0x7f24ac104460 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac113d90 con 0x7f24ac103ac0 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.873+0000 7f24b4600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac114090 con 0x7f24ac109ac0 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b2600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ac109ac0 0x7f24ac19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:37594/0 (socket says 172.21.15.3:37594) 2024-08-01T04:22:10.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 0x7f24ac19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 -- 172.21.15.3:0/3194276113 learned_addr learned my addr 172.21.15.3:0/3194276113 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 -- 172.21.15.3:0/3194276113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ac109ac0 msgr2=0x7f24ac19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ac109ac0 0x7f24ac19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:10.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 -- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 msgr2=0x7f24ac19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 0x7f24ac19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.874+0000 7f24b1c00640 1 -- 172.21.15.3:0/3194276113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac19f300 con 0x7f24ac103ac0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.875+0000 7f24b1c00640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac19a350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e9f0 tx=0x7f24a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.875+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000cd80 con 0x7f24ac103ac0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.875+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac19f5f0 con 0x7f24ac103ac0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.875+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a0010910 con 0x7f24ac103ac0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.875+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00188f0 con 0x7f24ac103ac0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.875+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1a6ed0 con 0x7f24ac103ac0 2024-08-01T04:22:10.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.876+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac103f20 con 0x7f24ac103ac0 2024-08-01T04:22:10.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.878+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f24a0010ac0 con 0x7f24ac103ac0 2024-08-01T04:22:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.880+0000 7f24aa000640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24940a2cf0 0x7f24940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.880+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f24a00cf830 con 0x7f24ac103ac0 2024-08-01T04:22:10.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.880+0000 7f24b1200640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24940a2cf0 0x7f24940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:10.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.881+0000 7f24b1200640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24940a2cf0 0x7f24940a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24980095f0 tx=0x7f2498005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:10.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:10.884+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f24a009bd60 con 0x7f24ac103ac0 2024-08-01T04:22:10.931 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:22:11.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.124+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24ac19b6b0 con 0x7f24ac103ac0 2024-08-01T04:22:11.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.125+0000 7f24aa000640 1 -- 172.21.15.3:0/3194276113 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f24a00a0060 con 0x7f24ac103ac0 2024-08-01T04:22:11.127 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:22:11.127 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:22:11.128 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.128+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24940a2cf0 msgr2=0x7f24940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.128+0000 7f24b4600640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24940a2cf0 0x7f24940a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24980095f0 tx=0x7f2498005f90 comp rx=0 tx=0).stop 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.128+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 msgr2=0x7f24ac19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.128+0000 7f24b4600640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac19a350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e9f0 tx=0x7f24a000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 shutdown_connections 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f24940a2cf0 0x7f24940a51b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ac109ac0 0x7f24ac19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ac104460 0x7f24ac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 --2- 172.21.15.3:0/3194276113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24ac103ac0 0x7f24ac19a350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 >> 172.21.15.3:0/3194276113 conn(0x7f24ac0ff7c0 msgr2=0x7f24ac0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 shutdown_connections 2024-08-01T04:22:11.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.129+0000 7f24b4600640 1 -- 172.21.15.3:0/3194276113 wait complete. 2024-08-01T04:22:11.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:11 smithi003 ceph-mon[32262]: pgmap v180: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.3 MiB/s wr, 372 op/s 2024-08-01T04:22:11.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:11 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3194276113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:22:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 -- 172.21.15.3:0/1772668898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81016f0 msgr2=0x7fbaf8101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 --2- 172.21.15.3:0/1772668898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81016f0 0x7fbaf8101b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009900 tx=0x7fbae402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 -- 172.21.15.3:0/1772668898 shutdown_connections 2024-08-01T04:22:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 --2- 172.21.15.3:0/1772668898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf8102090 0x7fbaf810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 --2- 172.21.15.3:0/1772668898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81016f0 0x7fbaf8101b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 --2- 172.21.15.3:0/1772668898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaf81076f0 0x7fbaf8107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 -- 172.21.15.3:0/1772668898 >> 172.21.15.3:0/1772668898 conn(0x7fbaf80fd470 msgr2=0x7fbaf80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 -- 172.21.15.3:0/1772668898 shutdown_connections 2024-08-01T04:22:11.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.719+0000 7fbafd600640 1 -- 172.21.15.3:0/1772668898 wait complete. 2024-08-01T04:22:11.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.720+0000 7fbafd600640 1 Processor -- start 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.720+0000 7fbafd600640 1 -- start start 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafd600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 0x7fbaf819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafd600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaf8102090 0x7fbaf819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafd600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81076f0 0x7fbaf819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafd600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8111b40 con 0x7fbaf81016f0 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafd600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81119c0 con 0x7fbaf81076f0 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafd600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8111cc0 con 0x7fbaf8102090 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 0x7fbaf819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 0x7fbaf819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55270/0 (socket says 172.21.15.3:55270) 2024-08-01T04:22:11.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 -- 172.21.15.3:0/3952727561 learned_addr learned my addr 172.21.15.3:0/3952727561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7400640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaf8102090 0x7fbaf819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbafcc00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81076f0 0x7fbaf819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 -- 172.21.15.3:0/3952727561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaf8102090 msgr2=0x7fbaf819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaf8102090 0x7fbaf819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 -- 172.21.15.3:0/3952727561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81076f0 msgr2=0x7fbaf819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.721+0000 7fbaf7e00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81076f0 0x7fbaf819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbaf7e00640 1 -- 172.21.15.3:0/3952727561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf819f160 con 0x7fbaf81016f0 2024-08-01T04:22:11.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbaf7e00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 0x7fbaf819a1b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbae800ebc0 tx=0x7fbae800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3952727561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8010630 con 0x7fbaf81016f0 2024-08-01T04:22:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf819f450 con 0x7fbaf81016f0 2024-08-01T04:22:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3952727561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae8010cb0 con 0x7fbaf81016f0 2024-08-01T04:22:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3952727561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8018bd0 con 0x7fbaf81016f0 2024-08-01T04:22:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.722+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a6d30 con 0x7fbaf81016f0 2024-08-01T04:22:11.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 Processor -- start 2024-08-01T04:22:11.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3952727561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fbae80107d0 con 0x7fbaf81016f0 2024-08-01T04:22:11.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 -- start start 2024-08-01T04:22:11.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fca14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a18fc06e30 con 0x55a18fcb4be0 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a18fb9ed40 con 0x55a18fca1a30 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.726+0000 7fbb01653740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a18fc02df0 con 0x55a18fca1110 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fca14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fca14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40640/0 (socket says 172.21.15.3:40640) 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 -- 172.21.15.3:0/2644919130 learned_addr learned my addr 172.21.15.3:0/2644919130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbaf7400640 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 -- 172.21.15.3:0/2644919130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 msgr2=0x55a18fcb46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 -- 172.21.15.3:0/2644919130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 msgr2=0x55a18fcb6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.727+0000 7fbafcc00640 1 -- 172.21.15.3:0/2644919130 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18f916580 con 0x55a18fca1110 2024-08-01T04:22:11.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbafcc00640 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fca14f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009ca0 tx=0x7fbaf0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b9443d637aee26 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:11.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbaed200640 1 -- 172.21.15.3:0/2644919130 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf001e070 con 0x55a18fca1110 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbaed200640 1 -- 172.21.15.3:0/2644919130 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf0009e50 con 0x55a18fca1110 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbaed200640 1 -- 172.21.15.3:0/2644919130 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000ea80 con 0x55a18fca1110 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbaf4c00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbadc0a2d70 0x7fbadc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbaf4c00640 1 -- 172.21.15.3:0/3952727561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbae80cf9a0 con 0x7fbaf81016f0 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.728+0000 7fbaf7e00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbadc0a2d70 0x7fbadc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 -- 172.21.15.3:0/2644919130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 msgr2=0x55a18fca14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fca14f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009ca0 tx=0x7fbaf0011810 comp rx=0 tx=0).stop 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 -- 172.21.15.3:0/2644919130 shutdown_connections 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 --2- 172.21.15.3:0/2644919130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fca14f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 -- 172.21.15.3:0/2644919130 >> 172.21.15.3:0/2644919130 conn(0x55a18fc9d780 msgr2=0x55a18fc9db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 -- 172.21.15.3:0/2644919130 shutdown_connections 2024-08-01T04:22:11.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbaf7e00640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbadc0a2d70 0x7fbadc0a5230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbae80cd160 tx=0x7fbae80d5550 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:11.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.729+0000 7fbb01653740 1 -- 172.21.15.3:0/2644919130 wait complete. 2024-08-01T04:22:11.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 -- start start 2024-08-01T04:22:11.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 Processor -- start 2024-08-01T04:22:11.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fcb1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a18fcb7750 con 0x55a18fcb4be0 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a18fbc1e70 con 0x55a18fca1a30 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a18fcb78d0 con 0x55a18fca1110 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.730+0000 7fbb01653740 10 client.0 _ll_register_callbacks cb 0x55a18fbb0f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fcb1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fcb1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40646/0 (socket says 172.21.15.3:40646) 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbafcc00640 1 -- 172.21.15.3:0/1591340563 learned_addr learned my addr 172.21.15.3:0/1591340563 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbafcc00640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbaf7400640 1 -- 172.21.15.3:0/1591340563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 msgr2=0x55a18fcb1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fcb1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbaf7400640 1 -- 172.21.15.3:0/1591340563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 msgr2=0x55a18fcb2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.731+0000 7fbaf7400640 1 -- 172.21.15.3:0/1591340563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18f9e0960 con 0x55a18fca1a30 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb2120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbae402a910 tx=0x7fbae40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f806a4db184b88da server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbae1600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbae1600640 1 -- 172.21.15.3:0/1591340563 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4035040 con 0x55a18fca1a30 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbae1600640 1 -- 172.21.15.3:0/1591340563 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbae4005450 con 0x55a18fca1a30 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf0009e50 con 0x55a18fca1a30 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbae1600640 1 -- 172.21.15.3:0/1591340563 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbae4005650 con 0x55a18fca1a30 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbb01653740 10 client.26033 fetch_fsmap learned FSMap version 29 2024-08-01T04:22:11.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.732+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a18fcc38f0 con 0x55a18fca1a30 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbae1600640 1 -- 172.21.15.3:0/1591340563 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf0009e50 con 0x55a18fca1a30 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbae1600640 1 -- 172.21.15.3:0/1591340563 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbae4003a10 con 0x55a18fca1a30 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbb01653740 10 client.26033 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbb01653740 10 client.26033 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbb01653740 20 client.26033 populate_metadata read hostname 'smithi003' 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbb01653740 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fcc4db0 0x55a18fcc71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbb01653740 4 client.26033 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.733+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18fc1a850 con 0x55a18fcc4db0 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.734+0000 7fbafcc00640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fcc4db0 0x55a18fcc71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.734+0000 7fbafcc00640 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fcc4db0 0x55a18fcc71a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95093671cb88775f server_cookie=8059e2e219178a87 in_seq=0 out_seq=0 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.734+0000 7fbae1600640 10 client.26033 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.735+0000 7fbae1600640 1 -- 172.21.15.3:0/1591340563 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a18fc1a850 con 0x55a18fcc4db0 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.735+0000 7fbae1600640 10 client.26033 handle_command_reply: tid=0 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.735+0000 7fbb01653740 1 client.26033 shutdown 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.735+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 msgr2=0x55a18fcb2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.735+0000 7fbb01653740 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb2120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbae402a910 tx=0x7fbae40043b0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.735+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 shutdown_connections 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fcc4db0 0x55a18fcc71a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb2120 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 --2- 172.21.15.3:0/1591340563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1110 0x55a18fcb1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 >> 172.21.15.3:0/1591340563 conn(0x55a18fc9d780 msgr2=0x55a18fc9c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbae1600640 0 client.26033 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 shutdown_connections 2024-08-01T04:22:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 1 -- 172.21.15.3:0/1591340563 wait complete. 2024-08-01T04:22:11.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.736+0000 7fbb01653740 20 client.26033 trim_cache size 0 max 16384 2024-08-01T04:22:11.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 Processor -- start 2024-08-01T04:22:11.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 -- start start 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1110 0x55a18fcb3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 0x55a18fccca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81adc80 con 0x55a18fcb4be0 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81ab550 con 0x55a18fca1110 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.755+0000 7fbb01653740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81a93e0 con 0x55a18fca1a30 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 0x55a18fccca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 0x55a18fccca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:40662/0 (socket says 172.21.15.3:40662) 2024-08-01T04:22:11.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 -- 172.21.15.3:0/1722422265 learned_addr learned my addr 172.21.15.3:0/1722422265 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbafcc00640 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1110 0x55a18fcb3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbafcc00640 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 -- 172.21.15.3:0/1722422265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1110 msgr2=0x55a18fcb3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1110 0x55a18fcb3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 -- 172.21.15.3:0/1722422265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 msgr2=0x55a18fcccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.756+0000 7fbaf7400640 1 -- 172.21.15.3:0/1722422265 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18fca1860 con 0x55a18fca1a30 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.757+0000 7fbaf7400640 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 0x55a18fccca70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbae4003e60 tx=0x7fbae4031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e32c082f32f71f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.757+0000 7fbae3e00640 1 -- 172.21.15.3:0/1722422265 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4035030 con 0x55a18fca1a30 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.757+0000 7fbae3e00640 1 -- 172.21.15.3:0/1722422265 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4041070 con 0x55a18fca1a30 2024-08-01T04:22:11.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.757+0000 7fbae3e00640 1 -- 172.21.15.3:0/1722422265 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4003850 con 0x55a18fca1a30 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 -- 172.21.15.3:0/1722422265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 msgr2=0x55a18fccca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 0x55a18fccca70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbae4003e60 tx=0x7fbae4031400 comp rx=0 tx=0).stop 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 -- 172.21.15.3:0/1722422265 shutdown_connections 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fcb4be0 0x55a18fcccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fca1a30 0x55a18fccca70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 --2- 172.21.15.3:0/1722422265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1110 0x55a18fcb3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 -- 172.21.15.3:0/1722422265 >> 172.21.15.3:0/1722422265 conn(0x55a18fc9d780 msgr2=0x55a18fcb3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 -- 172.21.15.3:0/1722422265 shutdown_connections 2024-08-01T04:22:11.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 -- 172.21.15.3:0/1722422265 wait complete. 2024-08-01T04:22:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.758+0000 7fbb01653740 1 -- start start 2024-08-01T04:22:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 Processor -- start 2024-08-01T04:22:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fca1110 0x55a18fcb8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fcb4be0 0x55a18fcb92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81a9a30 con 0x55a18fca1110 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81aae80 con 0x55a18fca1a30 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81aa0e0 con 0x55a18fcb4be0 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.759+0000 7fbb01653740 10 client.0 _ll_register_callbacks cb 0x7fbae4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fca1110 0x55a18fcb8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbafcc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fca1110 0x55a18fcb8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55304/0 (socket says 172.21.15.3:55304) 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbafcc00640 1 -- 172.21.15.3:0/2126037286 learned_addr learned my addr 172.21.15.3:0/2126037286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbafcc00640 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fcb4be0 0x55a18fcb92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 -- 172.21.15.3:0/2126037286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fcb4be0 msgr2=0x55a18fcb92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fcb4be0 0x55a18fcb92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 -- 172.21.15.3:0/2126037286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fca1110 msgr2=0x55a18fcb8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:22:11.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fca1110 0x55a18fcb8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 -- 172.21.15.3:0/2126037286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18fd2ea40 con 0x55a18fca1a30 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbaf7400640 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb8d60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a18fcb0310 tx=0x7fbae4031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04d3de73d047fed server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.760+0000 7fbae1600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.761+0000 7fbae1600640 1 -- 172.21.15.3:0/2126037286 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40037f0 con 0x55a18fca1a30 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.761+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbae4041070 con 0x55a18fca1a30 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.761+0000 7fbae1600640 1 -- 172.21.15.3:0/2126037286 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fbae4003990 con 0x55a18fca1a30 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.761+0000 7fbae1600640 1 -- 172.21.15.3:0/2126037286 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbae4067050 con 0x55a18fca1a30 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.761+0000 7fbb01653740 10 client.26036 fetch_fsmap learned FSMap version 29 2024-08-01T04:22:11.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.761+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a18fd2fcc0 con 0x55a18fca1a30 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbae1600640 1 -- 172.21.15.3:0/2126037286 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4041070 con 0x55a18fca1a30 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbae1600640 1 -- 172.21.15.3:0/2126037286 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbae4055020 con 0x55a18fca1a30 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbb01653740 10 client.26036 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbb01653740 10 client.26036 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbb01653740 20 client.26036 populate_metadata read hostname 'smithi003' 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbb01653740 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fd30320 0x55a18fd32710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbb01653740 4 client.26036 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b970358f", "release": true, "await": true}] 2024-08-01T04:22:11.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b970358f", "release": true, "await": true}) -- 0x55a18fc143a0 con 0x55a18fd30320 2024-08-01T04:22:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.762+0000 7fbafcc00640 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fd30320 0x55a18fd32710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:22:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.763+0000 7fbafcc00640 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fd30320 0x55a18fd32710 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92658aa1474d18e5 server_cookie=13ad9e8a8851095f in_seq=0 out_seq=0 2024-08-01T04:22:11.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.763+0000 7fbae1600640 10 client.26036 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:11.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:11 smithi163 ceph-mon[37383]: pgmap v180: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.3 MiB/s wr, 372 op/s 2024-08-01T04:22:11.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:11 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3194276113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:22:11.796 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 29, 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 16, 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "b970358f": { 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 16, 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:22:11.797 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 143.6, 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:22:11.798 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.786+0000 7fbae1600640 1 -- 172.21.15.3:0/2126037286 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a18fc143a0 con 0x55a18fd30320 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.786+0000 7fbae1600640 10 client.26036 handle_command_reply: tid=0 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.788+0000 7fbb01653740 1 client.26036 shutdown 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 msgr2=0x55a18fcb8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb8d60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a18fcb0310 tx=0x7fbae4031b10 comp rx=0 tx=0).stop 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 shutdown_connections 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a18fd30320 0x55a18fd32710 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a18fcb4be0 0x55a18fcb92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a18fca1a30 0x55a18fcb8d60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 --2- 172.21.15.3:0/2126037286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a18fca1110 0x55a18fcb8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbae1600640 0 client.26036 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:22:11.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 >> 172.21.15.3:0/2126037286 conn(0x55a18fc9d780 msgr2=0x55a18fcb3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 shutdown_connections 2024-08-01T04:22:11.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.790+0000 7fbb01653740 1 -- 172.21.15.3:0/2126037286 wait complete. 2024-08-01T04:22:11.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.791+0000 7fbb01653740 20 client.26036 trim_cache size 0 max 16384 2024-08-01T04:22:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.795+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbadc0a2d70 msgr2=0x7fbadc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.795+0000 7fbafd600640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbadc0a2d70 0x7fbadc0a5230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbae80cd160 tx=0x7fbae80d5550 comp rx=0 tx=0).stop 2024-08-01T04:22:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.795+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 msgr2=0x7fbaf819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:22:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.795+0000 7fbafd600640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 0x7fbaf819a1b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbae800ebc0 tx=0x7fbae800c6a0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 shutdown_connections 2024-08-01T04:22:11.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbaf81076f0 0x7fbaf819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbadc0a2d70 0x7fbadc0a5230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaf8102090 0x7fbaf819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 --2- 172.21.15.3:0/3952727561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbaf81016f0 0x7fbaf819a1b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:22:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 >> 172.21.15.3:0/3952727561 conn(0x7fbaf80fd470 msgr2=0x7fbaf810cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:22:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 shutdown_connections 2024-08-01T04:22:11.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:22:11.796+0000 7fbafd600640 1 -- 172.21.15.3:0/3952727561 wait complete. 2024-08-01T04:22:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:11 smithi160 ceph-mon[37224]: pgmap v180: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.3 MiB/s wr, 372 op/s 2024-08-01T04:22:11.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:11 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3194276113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:22:12.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:22:13.795 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:13 smithi163 ceph-mon[37383]: pgmap v181: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-08-01T04:22:13.797 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:13 smithi003 ceph-mon[32262]: pgmap v181: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-08-01T04:22:13.803 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b970358f', total seconds elapsed: 94.7 2024-08-01T04:22:13.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:13 smithi160 ceph-mon[37224]: pgmap v181: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-08-01T04:22:15.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:15 smithi160 ceph-mon[37224]: pgmap v182: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 588 KiB/s wr, 124 op/s 2024-08-01T04:22:15.924 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:15 smithi003 ceph-mon[32262]: pgmap v182: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 588 KiB/s wr, 124 op/s 2024-08-01T04:22:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:15 smithi163 ceph-mon[37383]: pgmap v182: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 588 KiB/s wr, 124 op/s 2024-08-01T04:22:16.071 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:22:17.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:17 smithi160 ceph-mon[37224]: pgmap v183: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.6 MiB/s wr, 501 op/s 2024-08-01T04:22:18.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:17 smithi163 ceph-mon[37383]: pgmap v183: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.6 MiB/s wr, 501 op/s 2024-08-01T04:22:18.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:17 smithi003 ceph-mon[32262]: pgmap v183: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.6 MiB/s wr, 501 op/s 2024-08-01T04:22:19.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:19 smithi160 ceph-mon[37224]: pgmap v184: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.9 MiB/s wr, 1.09k op/s 2024-08-01T04:22:20.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:19 smithi163 ceph-mon[37383]: pgmap v184: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.9 MiB/s wr, 1.09k op/s 2024-08-01T04:22:20.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:19 smithi003 ceph-mon[32262]: pgmap v184: 97 pgs: 97 active+clean; 22 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.9 MiB/s wr, 1.09k op/s 2024-08-01T04:22:21.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:21 smithi160 ceph-mon[37224]: pgmap v185: 97 pgs: 97 active+clean; 22 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 24 MiB/s wr, 4.11k op/s 2024-08-01T04:22:22.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:21 smithi163 ceph-mon[37383]: pgmap v185: 97 pgs: 97 active+clean; 22 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 24 MiB/s wr, 4.11k op/s 2024-08-01T04:22:22.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:21 smithi003 ceph-mon[32262]: pgmap v185: 97 pgs: 97 active+clean; 22 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 24 MiB/s wr, 4.11k op/s 2024-08-01T04:22:23.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:23 smithi160 ceph-mon[37224]: pgmap v186: 97 pgs: 97 active+clean; 22 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 27 MiB/s wr, 4.55k op/s 2024-08-01T04:22:24.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:23 smithi163 ceph-mon[37383]: pgmap v186: 97 pgs: 97 active+clean; 22 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 27 MiB/s wr, 4.55k op/s 2024-08-01T04:22:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:23 smithi003 ceph-mon[32262]: pgmap v186: 97 pgs: 97 active+clean; 22 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 27 MiB/s wr, 4.55k op/s 2024-08-01T04:22:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:25 smithi160 ceph-mon[37224]: pgmap v187: 97 pgs: 97 active+clean; 22 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 44 MiB/s wr, 7.36k op/s 2024-08-01T04:22:26.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:25 smithi163 ceph-mon[37383]: pgmap v187: 97 pgs: 97 active+clean; 22 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 44 MiB/s wr, 7.36k op/s 2024-08-01T04:22:26.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:25 smithi003 ceph-mon[32262]: pgmap v187: 97 pgs: 97 active+clean; 22 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 44 MiB/s wr, 7.36k op/s 2024-08-01T04:22:27.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:27 smithi160 ceph-mon[37224]: pgmap v188: 97 pgs: 97 active+clean; 22 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 49 MiB/s wr, 8.28k op/s 2024-08-01T04:22:28.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:27 smithi163 ceph-mon[37383]: pgmap v188: 97 pgs: 97 active+clean; 22 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 49 MiB/s wr, 8.28k op/s 2024-08-01T04:22:28.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:27 smithi003 ceph-mon[32262]: pgmap v188: 97 pgs: 97 active+clean; 22 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 49 MiB/s wr, 8.28k op/s 2024-08-01T04:22:29.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:29 smithi160 ceph-mon[37224]: pgmap v189: 97 pgs: 97 active+clean; 22 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 56 MiB/s wr, 9.38k op/s 2024-08-01T04:22:30.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:29 smithi163 ceph-mon[37383]: pgmap v189: 97 pgs: 97 active+clean; 22 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 56 MiB/s wr, 9.38k op/s 2024-08-01T04:22:30.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:29 smithi003 ceph-mon[32262]: pgmap v189: 97 pgs: 97 active+clean; 22 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 56 MiB/s wr, 9.38k op/s 2024-08-01T04:22:31.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:31 smithi160 ceph-mon[37224]: pgmap v190: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 77 MiB/s wr, 12.98k op/s 2024-08-01T04:22:32.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:31 smithi163 ceph-mon[37383]: pgmap v190: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 77 MiB/s wr, 12.98k op/s 2024-08-01T04:22:32.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:31 smithi003 ceph-mon[32262]: pgmap v190: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 77 MiB/s wr, 12.98k op/s 2024-08-01T04:22:33.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:33 smithi160 ceph-mon[37224]: pgmap v191: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 59 MiB/s wr, 9.96k op/s 2024-08-01T04:22:34.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:33 smithi163 ceph-mon[37383]: pgmap v191: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 59 MiB/s wr, 9.96k op/s 2024-08-01T04:22:34.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:33 smithi003 ceph-mon[32262]: pgmap v191: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 59 MiB/s wr, 9.96k op/s 2024-08-01T04:22:36.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:35 smithi163 ceph-mon[37383]: pgmap v192: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 68 MiB/s wr, 11.40k op/s 2024-08-01T04:22:36.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:35 smithi003 ceph-mon[32262]: pgmap v192: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 68 MiB/s wr, 11.40k op/s 2024-08-01T04:22:36.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:35 smithi160 ceph-mon[37224]: pgmap v192: 97 pgs: 97 active+clean; 23 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 68 MiB/s wr, 11.40k op/s 2024-08-01T04:22:37.954 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:37 smithi160 ceph-mon[37224]: pgmap v193: 97 pgs: 97 active+clean; 23 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 62 MiB/s wr, 10.32k op/s 2024-08-01T04:22:38.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:37 smithi163 ceph-mon[37383]: pgmap v193: 97 pgs: 97 active+clean; 23 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 62 MiB/s wr, 10.32k op/s 2024-08-01T04:22:38.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:37 smithi003 ceph-mon[32262]: pgmap v193: 97 pgs: 97 active+clean; 23 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 62 MiB/s wr, 10.32k op/s 2024-08-01T04:22:40.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:39 smithi163 ceph-mon[37383]: pgmap v194: 97 pgs: 97 active+clean; 23 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 63 MiB/s wr, 10.36k op/s 2024-08-01T04:22:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:39 smithi003 ceph-mon[32262]: pgmap v194: 97 pgs: 97 active+clean; 23 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 63 MiB/s wr, 10.36k op/s 2024-08-01T04:22:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:39 smithi160 ceph-mon[37224]: pgmap v194: 97 pgs: 97 active+clean; 23 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 63 MiB/s wr, 10.36k op/s 2024-08-01T04:22:42.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:41 smithi163 ceph-mon[37383]: pgmap v195: 97 pgs: 97 active+clean; 23 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 67 MiB/s wr, 10.86k op/s 2024-08-01T04:22:42.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:41 smithi003 ceph-mon[32262]: pgmap v195: 97 pgs: 97 active+clean; 23 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 67 MiB/s wr, 10.86k op/s 2024-08-01T04:22:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:41 smithi160 ceph-mon[37224]: pgmap v195: 97 pgs: 97 active+clean; 23 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 67 MiB/s wr, 10.86k op/s 2024-08-01T04:22:44.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:43 smithi163 ceph-mon[37383]: pgmap v196: 97 pgs: 97 active+clean; 23 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 50 MiB/s wr, 7.88k op/s 2024-08-01T04:22:44.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:43 smithi003 ceph-mon[32262]: pgmap v196: 97 pgs: 97 active+clean; 23 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 50 MiB/s wr, 7.88k op/s 2024-08-01T04:22:44.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:43 smithi160 ceph-mon[37224]: pgmap v196: 97 pgs: 97 active+clean; 23 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 50 MiB/s wr, 7.88k op/s 2024-08-01T04:22:46.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:45 smithi163 ceph-mon[37383]: pgmap v197: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 60 MiB/s wr, 9.44k op/s 2024-08-01T04:22:46.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:45 smithi003 ceph-mon[32262]: pgmap v197: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 60 MiB/s wr, 9.44k op/s 2024-08-01T04:22:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:45 smithi160 ceph-mon[37224]: pgmap v197: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 60 MiB/s wr, 9.44k op/s 2024-08-01T04:22:48.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:47 smithi163 ceph-mon[37383]: pgmap v198: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 61 MiB/s wr, 9.65k op/s 2024-08-01T04:22:48.051 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:47 smithi003 ceph-mon[32262]: pgmap v198: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 61 MiB/s wr, 9.65k op/s 2024-08-01T04:22:48.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:47 smithi160 ceph-mon[37224]: pgmap v198: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 61 MiB/s wr, 9.65k op/s 2024-08-01T04:22:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:49 smithi163 ceph-mon[37383]: pgmap v199: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 52 MiB/s wr, 8.17k op/s 2024-08-01T04:22:50.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:49 smithi003 ceph-mon[32262]: pgmap v199: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 52 MiB/s wr, 8.17k op/s 2024-08-01T04:22:50.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:49 smithi160 ceph-mon[37224]: pgmap v199: 97 pgs: 97 active+clean; 23 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 52 MiB/s wr, 8.17k op/s 2024-08-01T04:22:52.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:51 smithi163 ceph-mon[37383]: pgmap v200: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 64 MiB/s wr, 9.87k op/s 2024-08-01T04:22:52.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:51 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:22:52.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:51 smithi003 ceph-mon[32262]: pgmap v200: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 64 MiB/s wr, 9.87k op/s 2024-08-01T04:22:52.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:51 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:22:52.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:51 smithi160 ceph-mon[37224]: pgmap v200: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 64 MiB/s wr, 9.87k op/s 2024-08-01T04:22:52.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:51 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:22:54.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:53 smithi163 ceph-mon[37383]: pgmap v201: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 53 MiB/s wr, 8.25k op/s 2024-08-01T04:22:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:53 smithi003 ceph-mon[32262]: pgmap v201: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 53 MiB/s wr, 8.25k op/s 2024-08-01T04:22:54.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:53 smithi160 ceph-mon[37224]: pgmap v201: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 53 MiB/s wr, 8.25k op/s 2024-08-01T04:22:56.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:55 smithi163 ceph-mon[37383]: pgmap v202: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 50 MiB/s wr, 7.78k op/s 2024-08-01T04:22:56.052 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:55 smithi003 ceph-mon[32262]: pgmap v202: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 50 MiB/s wr, 7.78k op/s 2024-08-01T04:22:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:55 smithi160 ceph-mon[37224]: pgmap v202: 97 pgs: 97 active+clean; 24 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 50 MiB/s wr, 7.78k op/s 2024-08-01T04:22:58.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:57 smithi163 ceph-mon[37383]: pgmap v203: 97 pgs: 97 active+clean; 24 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 62 MiB/s wr, 9.97k op/s 2024-08-01T04:22:58.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:57 smithi003 ceph-mon[32262]: pgmap v203: 97 pgs: 97 active+clean; 24 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 62 MiB/s wr, 9.97k op/s 2024-08-01T04:22:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:57 smithi160 ceph-mon[37224]: pgmap v203: 97 pgs: 97 active+clean; 24 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 62 MiB/s wr, 9.97k op/s 2024-08-01T04:23:00.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:22:59 smithi163 ceph-mon[37383]: pgmap v204: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 53 MiB/s wr, 8.39k op/s 2024-08-01T04:23:00.050 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:22:59 smithi003 ceph-mon[32262]: pgmap v204: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 53 MiB/s wr, 8.39k op/s 2024-08-01T04:23:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:22:59 smithi160 ceph-mon[37224]: pgmap v204: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 53 MiB/s wr, 8.39k op/s 2024-08-01T04:23:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 -- 172.21.15.3:0/485000906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac10e000 msgr2=0x7fc2ac1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 --2- 172.21.15.3:0/485000906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac10e000 0x7fc2ac1103f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009ca0 tx=0x7fc2a402a8c0 comp rx=0 tx=0).stop 2024-08-01T04:23:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 -- 172.21.15.3:0/485000906 shutdown_connections 2024-08-01T04:23:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 --2- 172.21.15.3:0/485000906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac10e000 0x7fc2ac1103f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:01.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 --2- 172.21.15.3:0/485000906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac100e00 0x7fc2ac10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 --2- 172.21.15.3:0/485000906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100450 0x7fc2ac100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 -- 172.21.15.3:0/485000906 >> 172.21.15.3:0/485000906 conn(0x7fc2ac0fc340 msgr2=0x7fc2ac0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 -- 172.21.15.3:0/485000906 shutdown_connections 2024-08-01T04:23:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.896+0000 7fc2abe00640 1 -- 172.21.15.3:0/485000906 wait complete. 2024-08-01T04:23:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.897+0000 7fc2abe00640 1 Processor -- start 2024-08-01T04:23:01.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.897+0000 7fc2abe00640 1 -- start start 2024-08-01T04:23:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.897+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac100450 0x7fc2ac19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100e00 0x7fc2ac19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2abe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac10e000 0x7fc2ac19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2abe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac113d00 con 0x7fc2ac100450 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2abe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac113b80 con 0x7fc2ac10e000 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2abe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac113e80 con 0x7fc2ac100e00 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac100450 0x7fc2ac19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac10e000 0x7fc2ac19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac100450 0x7fc2ac19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52602/0 (socket says 172.21.15.3:52602) 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2aaa00640 1 -- 172.21.15.3:0/2860023839 learned_addr learned my addr 172.21.15.3:0/2860023839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2aa000640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100e00 0x7fc2ac19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2ab400640 1 -- 172.21.15.3:0/2860023839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100e00 msgr2=0x7fc2ac19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2ab400640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100e00 0x7fc2ac19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2ab400640 1 -- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac100450 msgr2=0x7fc2ac19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2ab400640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac100450 0x7fc2ac19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2ab400640 1 -- 172.21.15.3:0/2860023839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac19f1f0 con 0x7fc2ac10e000 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.898+0000 7fc2aa000640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100e00 0x7fc2ac19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.899+0000 7fc2ab400640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac10e000 0x7fc2ac19eb10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc2a402ade0 tx=0x7fc2a40319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.899+0000 7fc2a3400640 1 -- 172.21.15.3:0/2860023839 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4003b70 con 0x7fc2ac10e000 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.899+0000 7fc2a3400640 1 -- 172.21.15.3:0/2860023839 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a4003d10 con 0x7fc2ac10e000 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.899+0000 7fc2a3400640 1 -- 172.21.15.3:0/2860023839 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a402fc10 con 0x7fc2ac10e000 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.899+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac19f480 con 0x7fc2ac10e000 2024-08-01T04:23:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.899+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1a6dc0 con 0x7fc2ac10e000 2024-08-01T04:23:01.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.902+0000 7fc2a3400640 1 -- 172.21.15.3:0/2860023839 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc2a4004050 con 0x7fc2ac10e000 2024-08-01T04:23:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.903+0000 7fc2a3400640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc2900a2d70 0x7fc2900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.904+0000 7fc2a3400640 1 -- 172.21.15.3:0/2860023839 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc2a40f0390 con 0x7fc2ac10e000 2024-08-01T04:23:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.904+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc2900a2d70 0x7fc2900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:01.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:01.904+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc2900a2d70 0x7fc2900a5230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009cd0 tx=0x7fc29c005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:02.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:01 smithi163 ceph-mon[37383]: pgmap v205: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 65 MiB/s wr, 10.26k op/s 2024-08-01T04:23:02.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:01 smithi003 ceph-mon[32262]: pgmap v205: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 65 MiB/s wr, 10.26k op/s 2024-08-01T04:23:02.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:01 smithi160 ceph-mon[37224]: pgmap v205: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 65 MiB/s wr, 10.26k op/s 2024-08-01T04:23:02.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.565+0000 7fc2b420b740 1 Processor -- start 2024-08-01T04:23:02.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.565+0000 7fc2b420b740 1 -- start start 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 0x55a17c5dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c5e7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2b420b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a17c541bf0 con 0x55a17c5dd000 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2b420b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a17c4d9b20 con 0x55a17c5e78c0 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2b420b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a17c53dbb0 con 0x55a17c5dc6e0 2024-08-01T04:23:02.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 0x55a17c5dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.566+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 0x55a17c5dcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42730/0 (socket says 172.21.15.3:42730) 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aaa00640 1 -- 172.21.15.3:0/2329497772 learned_addr learned my addr 172.21.15.3:0/2329497772 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aa000640 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c5e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2ab400640 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aaa00640 1 -- 172.21.15.3:0/2329497772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 msgr2=0x55a17c5e9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aaa00640 1 -- 172.21.15.3:0/2329497772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 msgr2=0x55a17c5e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c5e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.567+0000 7fc2aaa00640 1 -- 172.21.15.3:0/2329497772 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a17c4e07f0 con 0x55a17c5dc6e0 2024-08-01T04:23:02.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.568+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 0x55a17c5dcac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc29c01c410 tx=0x7fc29c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81fa75703c80933e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:02.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.568+0000 7fc297e00640 1 -- 172.21.15.3:0/2329497772 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c02b070 con 0x55a17c5dc6e0 2024-08-01T04:23:02.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.568+0000 7fc297e00640 1 -- 172.21.15.3:0/2329497772 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c026b00 con 0x55a17c5dc6e0 2024-08-01T04:23:02.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.568+0000 7fc297e00640 1 -- 172.21.15.3:0/2329497772 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c01abc0 con 0x55a17c5dc6e0 2024-08-01T04:23:02.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 -- 172.21.15.3:0/2329497772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 msgr2=0x55a17c5dcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:02.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 0x55a17c5dcac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc29c01c410 tx=0x7fc29c014000 comp rx=0 tx=0).stop 2024-08-01T04:23:02.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 -- 172.21.15.3:0/2329497772 shutdown_connections 2024-08-01T04:23:02.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c5e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 --2- 172.21.15.3:0/2329497772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dc6e0 0x55a17c5dcac0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 -- 172.21.15.3:0/2329497772 >> 172.21.15.3:0/2329497772 conn(0x55a17c5d8c70 msgr2=0x55a17c5d9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 -- 172.21.15.3:0/2329497772 shutdown_connections 2024-08-01T04:23:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.569+0000 7fc2b420b740 1 -- 172.21.15.3:0/2329497772 wait complete. 2024-08-01T04:23:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.570+0000 7fc2b420b740 1 -- start start 2024-08-01T04:23:02.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.571+0000 7fc2b420b740 1 Processor -- start 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.571+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.571+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5e78c0 0x55a17c5e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2b420b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a17c5ea490 con 0x55a17c5e78c0 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2b420b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a17c4fccf0 con 0x55a17c5dc6e0 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2b420b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a17c5ea610 con 0x55a17c5dd000 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2b420b740 10 client.0 _ll_register_callbacks cb 0x55a17c4e8e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60940/0 (socket says 172.21.15.3:60940) 2024-08-01T04:23:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 -- 172.21.15.3:0/750845807 learned_addr learned my addr 172.21.15.3:0/750845807 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aa000640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2ab400640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5e78c0 0x55a17c5e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 -- 172.21.15.3:0/750845807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 msgr2=0x55a17c5e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 -- 172.21.15.3:0/750845807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5e78c0 msgr2=0x55a17c5e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5e78c0 0x55a17c5e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.572+0000 7fc2aaa00640 1 -- 172.21.15.3:0/750845807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a17c2b33c0 con 0x55a17c5dc6e0 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.573+0000 7fc2aa000640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.573+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc29c01a860 tx=0x7fc29c01a390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2ff3395b55874b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.573+0000 7fc294c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:23:02.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.573+0000 7fc294c00640 1 -- 172.21.15.3:0/750845807 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c036060 con 0x55a17c5dc6e0 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.573+0000 7fc294c00640 1 -- 172.21.15.3:0/750845807 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc29c009040 con 0x55a17c5dc6e0 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.573+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc29c026b00 con 0x55a17c5dc6e0 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc294c00640 1 -- 172.21.15.3:0/750845807 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc29c02b030 con 0x55a17c5dc6e0 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 10 client.26048 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a17c5f61c0 con 0x55a17c5dc6e0 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc294c00640 1 -- 172.21.15.3:0/750845807 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c026b00 con 0x55a17c5dc6e0 2024-08-01T04:23:02.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc294c00640 1 -- 172.21.15.3:0/750845807 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc29c005200 con 0x55a17c5dc6e0 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 10 client.26048 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 10 client.26048 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 20 client.26048 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5f7510 0x55a17c5f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 4 client.26048 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.574+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a17c54f160 con 0x55a17c5f7510 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.575+0000 7fc2aa000640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5f7510 0x55a17c5f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.575+0000 7fc2aa000640 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5f7510 0x55a17c5f9900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3ee53c1d7c3f65e server_cookie=11c581c4347f2289 in_seq=0 out_seq=0 2024-08-01T04:23:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.575+0000 7fc294c00640 10 client.26048 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:23:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.576+0000 7fc294c00640 1 -- 172.21.15.3:0/750845807 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a17c54f160 con 0x55a17c5f7510 2024-08-01T04:23:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.576+0000 7fc294c00640 10 client.26048 handle_command_reply: tid=0 2024-08-01T04:23:02.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.576+0000 7fc2b420b740 1 client.26048 shutdown 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 msgr2=0x55a17c5e4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc29c01a860 tx=0x7fc29c01a390 comp rx=0 tx=0).stop 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 shutdown_connections 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5e78c0 0x55a17c5e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5f7510 0x55a17c5f9900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 --2- 172.21.15.3:0/750845807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4840 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 >> 172.21.15.3:0/750845807 conn(0x55a17c5d8c70 msgr2=0x55a17c5de9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc294c00640 0 client.26048 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:23:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 shutdown_connections 2024-08-01T04:23:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 1 -- 172.21.15.3:0/750845807 wait complete. 2024-08-01T04:23:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.577+0000 7fc2b420b740 20 client.26048 trim_cache size 0 max 16384 2024-08-01T04:23:02.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.605+0000 7fc2b420b740 1 Processor -- start 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.606+0000 7fc2b420b740 1 -- start start 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.606+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.606+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.606+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2aaa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2b420b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1b2060 con 0x55a17c5dc6e0 2024-08-01T04:23:02.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2b420b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1af930 con 0x55a17c5e78c0 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2b420b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ad780 con 0x55a17c5dd000 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2aa000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2aa000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42746/0 (socket says 172.21.15.3:42746) 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.607+0000 7fc2aa000640 1 -- 172.21.15.3:0/775180423 learned_addr learned my addr 172.21.15.3:0/775180423 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.608+0000 7fc2aa000640 1 -- 172.21.15.3:0/775180423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 msgr2=0x55a17c5e3650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.608+0000 7fc2aa000640 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.608+0000 7fc2aa000640 1 -- 172.21.15.3:0/775180423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dc6e0 msgr2=0x55a17c5e4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.608+0000 7fc2aa000640 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.608+0000 7fc2aa000640 1 -- 172.21.15.3:0/775180423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a17c5d71f0 con 0x55a17c5dd000 2024-08-01T04:23:02.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.608+0000 7fc2aa000640 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e3110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55a17c5e40d0 tx=0x7fc298005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2818e49256de5b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:02.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.609+0000 7fc297400640 1 -- 172.21.15.3:0/775180423 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298012e60 con 0x55a17c5dd000 2024-08-01T04:23:02.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.609+0000 7fc297400640 1 -- 172.21.15.3:0/775180423 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298012110 con 0x55a17c5dd000 2024-08-01T04:23:02.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.609+0000 7fc297400640 1 -- 172.21.15.3:0/775180423 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980047a0 con 0x55a17c5dd000 2024-08-01T04:23:02.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 -- 172.21.15.3:0/775180423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 msgr2=0x55a17c5e3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:02.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e3110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55a17c5e40d0 tx=0x7fc298005f90 comp rx=0 tx=0).stop 2024-08-01T04:23:02.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 -- 172.21.15.3:0/775180423 shutdown_connections 2024-08-01T04:23:02.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5e78c0 0x55a17c5e3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5dd000 0x55a17c5e3110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 --2- 172.21.15.3:0/775180423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dc6e0 0x55a17c5e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.610+0000 7fc2b420b740 1 -- 172.21.15.3:0/775180423 >> 172.21.15.3:0/775180423 conn(0x55a17c5d8c70 msgr2=0x55a17c5dfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:02.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.613+0000 7fc2b420b740 1 -- 172.21.15.3:0/775180423 shutdown_connections 2024-08-01T04:23:02.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.613+0000 7fc2b420b740 1 -- 172.21.15.3:0/775180423 wait complete. 2024-08-01T04:23:02.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.614+0000 7fc2b420b740 1 -- start start 2024-08-01T04:23:02.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.614+0000 7fc2b420b740 1 Processor -- start 2024-08-01T04:23:02.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c665d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c666280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 0x55a17c6629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1addf0 con 0x55a17c5dd000 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1af260 con 0x55a17c5dc6e0 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1ae4c0 con 0x55a17c5e78c0 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.615+0000 7fc2b420b740 10 client.0 _ll_register_callbacks cb 0x7fc2a4003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 0x55a17c6629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 0x55a17c6629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42758/0 (socket says 172.21.15.3:42758) 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2ab400640 1 -- 172.21.15.3:0/1651385950 learned_addr learned my addr 172.21.15.3:0/1651385950 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:02.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c665d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2aa000640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c666280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2ab400640 1 -- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 msgr2=0x55a17c665d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.616+0000 7fc2ab400640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c665d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.617+0000 7fc2ab400640 1 -- 172.21.15.3:0/1651385950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 msgr2=0x55a17c666280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:02.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.617+0000 7fc2ab400640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c666280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.617+0000 7fc2ab400640 1 -- 172.21.15.3:0/1651385950 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a17c663220 con 0x55a17c5e78c0 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.617+0000 7fc2ab400640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 0x55a17c6629f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40b8ea0 tx=0x7fc2a40bc810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=732824b554653b55 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.617+0000 7fc294c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.618+0000 7fc2aa000640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c666280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.618+0000 7fc294c00640 1 -- 172.21.15.3:0/1651385950 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a402fdb0 con 0x55a17c5e78c0 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.618+0000 7fc294c00640 1 -- 172.21.15.3:0/1651385950 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc2a40db350 con 0x55a17c5e78c0 2024-08-01T04:23:02.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.618+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc298012110 con 0x55a17c5e78c0 2024-08-01T04:23:02.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.618+0000 7fc294c00640 1 -- 172.21.15.3:0/1651385950 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2a40bd140 con 0x55a17c5e78c0 2024-08-01T04:23:02.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.618+0000 7fc2b420b740 10 client.26038 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:02.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.619+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a17c5ef4e0 con 0x55a17c5e78c0 2024-08-01T04:23:02.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.619+0000 7fc294c00640 1 -- 172.21.15.3:0/1651385950 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc298012110 con 0x55a17c5e78c0 2024-08-01T04:23:02.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc294c00640 1 -- 172.21.15.3:0/1651385950 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc2a40bcca0 con 0x55a17c5e78c0 2024-08-01T04:23:02.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc2b420b740 10 client.26038 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:02.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc2b420b740 10 client.26038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:23:02.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc2b420b740 20 client.26038 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:02.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc2b420b740 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5efb60 0x55a17c5eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:02.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc2b420b740 4 client.26038 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:23:02.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.621+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a17c549670 con 0x55a17c5efb60 2024-08-01T04:23:02.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.622+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5efb60 0x55a17c5eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.623+0000 7fc2aaa00640 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5efb60 0x55a17c5eff40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7b82383f56a2d82 server_cookie=b14d04682987411e in_seq=0 out_seq=0 2024-08-01T04:23:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.623+0000 7fc294c00640 10 client.26038 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:23:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.623+0000 7fc294c00640 1 -- 172.21.15.3:0/1651385950 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a17c549670 con 0x55a17c5efb60 2024-08-01T04:23:02.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.623+0000 7fc294c00640 10 client.26038 handle_command_reply: tid=0 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.623+0000 7fc2b420b740 1 client.26038 shutdown 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 msgr2=0x55a17c6629f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 0x55a17c6629f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40b8ea0 tx=0x7fc2a40bc810 comp rx=0 tx=0).stop 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 shutdown_connections 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a17c5efb60 0x55a17c5eff40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a17c5e78c0 0x55a17c6629f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a17c5dd000 0x55a17c666280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 --2- 172.21.15.3:0/1651385950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a17c5dc6e0 0x55a17c665d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc294c00640 0 client.26038 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:23:02.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.624+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 >> 172.21.15.3:0/1651385950 conn(0x55a17c5d8c70 msgr2=0x55a17c5dfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:02.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.625+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 shutdown_connections 2024-08-01T04:23:02.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.625+0000 7fc2b420b740 1 -- 172.21.15.3:0/1651385950 wait complete. 2024-08-01T04:23:02.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.626+0000 7fc2b420b740 20 client.26038 trim_cache size 0 max 16384 2024-08-01T04:23:02.628 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:23:02.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.627+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc2900a2d70 msgr2=0x7fc2900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:02.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.627+0000 7fc2abe00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc2900a2d70 0x7fc2900a5230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009cd0 tx=0x7fc29c005f90 comp rx=0 tx=0).stop 2024-08-01T04:23:02.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.627+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac10e000 msgr2=0x7fc2ac19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:02.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.627+0000 7fc2abe00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac10e000 0x7fc2ac19eb10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc2a402ade0 tx=0x7fc2a40319f0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.628+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 shutdown_connections 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.628+0000 7fc2abe00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2ac10e000 0x7fc2ac19eb10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.628+0000 7fc2abe00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc2900a2d70 0x7fc2900a5230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.628+0000 7fc2abe00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2ac100e00 0x7fc2ac19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.628+0000 7fc2abe00640 1 --2- 172.21.15.3:0/2860023839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2ac100450 0x7fc2ac19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.628+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 >> 172.21.15.3:0/2860023839 conn(0x7fc2ac0fc340 msgr2=0x7fc2ac0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.629+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 shutdown_connections 2024-08-01T04:23:02.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:02.629+0000 7fc2abe00640 1 -- 172.21.15.3:0/2860023839 wait complete. 2024-08-01T04:23:04.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:03 smithi163 ceph-mon[37383]: pgmap v206: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 53 MiB/s wr, 8.49k op/s 2024-08-01T04:23:04.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:03 smithi003 ceph-mon[32262]: pgmap v206: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 53 MiB/s wr, 8.49k op/s 2024-08-01T04:23:04.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:03 smithi160 ceph-mon[37224]: pgmap v206: 97 pgs: 97 active+clean; 24 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 53 MiB/s wr, 8.49k op/s 2024-08-01T04:23:06.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:05 smithi163 ceph-mon[37383]: pgmap v207: 97 pgs: 97 active+clean; 24 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 55 MiB/s wr, 8.89k op/s 2024-08-01T04:23:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:05 smithi003 ceph-mon[32262]: pgmap v207: 97 pgs: 97 active+clean; 24 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 55 MiB/s wr, 8.89k op/s 2024-08-01T04:23:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:05 smithi160 ceph-mon[37224]: pgmap v207: 97 pgs: 97 active+clean; 24 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 55 MiB/s wr, 8.89k op/s 2024-08-01T04:23:08.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:07 smithi163 ceph-mon[37383]: pgmap v208: 97 pgs: 97 active+clean; 24 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 74 MiB/s wr, 11.55k op/s 2024-08-01T04:23:08.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:07 smithi003 ceph-mon[32262]: pgmap v208: 97 pgs: 97 active+clean; 24 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 74 MiB/s wr, 11.55k op/s 2024-08-01T04:23:08.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:07 smithi160 ceph-mon[37224]: pgmap v208: 97 pgs: 97 active+clean; 24 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 74 MiB/s wr, 11.55k op/s 2024-08-01T04:23:10.118 INFO:tasks.workunit.client.0.smithi003.stdout:Wrote -1 instead of 4096 bytes. 2024-08-01T04:23:10.118 INFO:tasks.workunit.client.0.smithi003.stdout:Probably out of disk space 2024-08-01T04:23:10.118 INFO:tasks.workunit.client.0.smithi003.stderr:write: Resource temporarily unavailable 2024-08-01T04:23:10.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:09 smithi160 ceph-mon[37224]: pgmap v209: 97 pgs: 97 active+clean; 24 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 53 MiB/s wr, 8.13k op/s 2024-08-01T04:23:10.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:09 smithi163 ceph-mon[37383]: pgmap v209: 97 pgs: 97 active+clean; 24 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 53 MiB/s wr, 8.13k op/s 2024-08-01T04:23:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:09 smithi003 ceph-mon[32262]: pgmap v209: 97 pgs: 97 active+clean; 24 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 53 MiB/s wr, 8.13k op/s 2024-08-01T04:23:11.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:23:11.570 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-08-01T04:23:11.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-01T04:23:12.136 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:11 smithi003 ceph-mon[32262]: pgmap v210: 97 pgs: 97 active+clean; 25 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 70 MiB/s wr, 10.47k op/s 2024-08-01T04:23:12.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:11 smithi160 ceph-mon[37224]: pgmap v210: 97 pgs: 97 active+clean; 25 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 70 MiB/s wr, 10.47k op/s 2024-08-01T04:23:12.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:11 smithi163 ceph-mon[37383]: pgmap v210: 97 pgs: 97 active+clean; 25 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 70 MiB/s wr, 10.47k op/s 2024-08-01T04:23:13.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:12 smithi160 ceph-mon[37224]: osdmap e92: 12 total, 12 up, 12 in 2024-08-01T04:23:13.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:12 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:23:13.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:12 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:23:13.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:12 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:23:13.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:12 smithi163 ceph-mon[37383]: osdmap e92: 12 total, 12 up, 12 in 2024-08-01T04:23:13.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:12 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:23:13.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:12 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:23:13.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:12 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:23:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:12 smithi003 ceph-mon[32262]: osdmap e92: 12 total, 12 up, 12 in 2024-08-01T04:23:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:12 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:23:13.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:12 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:23:13.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:12 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:23:13.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:23:14.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:13 smithi003 ceph-mon[32262]: pgmap v212: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 69 MiB/s wr, 9.98k op/s 2024-08-01T04:23:14.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:13 smithi160 ceph-mon[37224]: pgmap v212: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 69 MiB/s wr, 9.98k op/s 2024-08-01T04:23:14.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:13 smithi163 ceph-mon[37383]: pgmap v212: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 69 MiB/s wr, 9.98k op/s 2024-08-01T04:23:14.309 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:23:15.535 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d99c9769be1521395979e8c23d9073adbf1ff43f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-08-01T04:23:16.328 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=035c9d7ff720453d974831aa65054c94 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d99c9769be1521395979e8c23d9073adbf1ff43f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-08-01T04:23:16.331 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-01T04:23:16.341 INFO:tasks.quiescer:joining Quiescers 2024-08-01T04:23:16.341 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-01T04:23:16.341 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:23:16.346 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:15 smithi163 ceph-mon[37383]: pgmap v213: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 66 MiB/s wr, 9.43k op/s 2024-08-01T04:23:16.346 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:15 smithi160 ceph-mon[37224]: pgmap v213: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 66 MiB/s wr, 9.43k op/s 2024-08-01T04:23:16.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.616+0000 7f4965e00640 1 -- 172.21.15.3:0/2170631931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 msgr2=0x7f49601166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f495d600640 1 -- 172.21.15.3:0/2170631931 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948037070 con 0x7f4960109840 2024-08-01T04:23:16.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 --2- 172.21.15.3:0/2170631931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601166a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4948009890 tx=0x7f494802a3b0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 -- 172.21.15.3:0/2170631931 shutdown_connections 2024-08-01T04:23:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 --2- 172.21.15.3:0/2170631931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960116be0 0x7f4960118fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 --2- 172.21.15.3:0/2170631931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601166a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 --2- 172.21.15.3:0/2170631931 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4960108f20 0x7f4960109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 -- 172.21.15.3:0/2170631931 >> 172.21.15.3:0/2170631931 conn(0x7f496006e000 msgr2=0x7f496006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:16.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 -- 172.21.15.3:0/2170631931 shutdown_connections 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.618+0000 7f4965e00640 1 -- 172.21.15.3:0/2170631931 wait complete. 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.620+0000 7f4965e00640 1 Processor -- start 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.624+0000 7f4965e00640 1 -- start start 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.625+0000 7f4965e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 0x7f49601a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.625+0000 7f4965e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601a8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.625+0000 7f4965e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4960116be0 0x7f49601a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.625+0000 7f495ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 0x7f49601a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.625+0000 7f495ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 0x7f49601a8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50398/0 (socket says 172.21.15.3:50398) 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.625+0000 7f4965e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f496011c6c0 con 0x7f4960116be0 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.626+0000 7f4965e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f496011c540 con 0x7f4960108f20 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.626+0000 7f4965e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f496011c840 con 0x7f4960109840 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.626+0000 7f495ea00640 1 -- 172.21.15.3:0/24927191 learned_addr learned my addr 172.21.15.3:0/24927191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.626+0000 7f495e000640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601a8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.626+0000 7f495f400640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4960116be0 0x7f49601a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.627+0000 7f495ea00640 1 -- 172.21.15.3:0/24927191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 msgr2=0x7f49601a8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.627+0000 7f495ea00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601a8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.627+0000 7f495ea00640 1 -- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4960116be0 msgr2=0x7f49601a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.627+0000 7f495ea00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4960116be0 0x7f49601a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.627+0000 7f495ea00640 1 -- 172.21.15.3:0/24927191 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49601a2fa0 con 0x7f4960108f20 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.627+0000 7f495e000640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601a8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.628+0000 7f495ea00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 0x7f49601a8640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9c0 tx=0x7f495000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:16.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.629+0000 7f4964a00640 1 -- 172.21.15.3:0/24927191 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495000ccc0 con 0x7f4960108f20 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.629+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49601a3290 con 0x7f4960108f20 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.630+0000 7f4964a00640 1 -- 172.21.15.3:0/24927191 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4950005500 con 0x7f4960108f20 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.630+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49601af5c0 con 0x7f4960108f20 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.632+0000 7f4964a00640 1 -- 172.21.15.3:0/24927191 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950018430 con 0x7f4960108f20 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.641+0000 7f4964a00640 1 -- 172.21.15.3:0/24927191 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f49500046c0 con 0x7f4960108f20 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.643+0000 7f4964a00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49440a2db0 0x7f49440a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.645+0000 7f495e000640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49440a2db0 0x7f49440a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.649+0000 7f4964a00640 1 -- 172.21.15.3:0/24927191 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8673+0+0 (secure 0 0 0) 0x7f49500cfa00 con 0x7f4960108f20 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.649+0000 7f495e000640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49440a2db0 0x7f49440a5270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49480057c0 tx=0x7f494800b040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.654+0000 7f4968522740 1 Processor -- start 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.655+0000 7f4968522740 1 -- start start 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.656+0000 7f4968522740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 0x561525a8e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.656+0000 7f4968522740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f4968522740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a9bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f495e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f4968522740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5615259f3bf0 con 0x561525a996d0 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f4968522740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x56152598bb20 con 0x561525a8ee10 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f4968522740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5615259efbb0 con 0x561525a8e4f0 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f495ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 0x561525a8e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f495e000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a99190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50404/0 (socket says 172.21.15.3:50404) 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f495ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 0x561525a8e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:44316/0 (socket says 172.21.15.3:44316) 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f495ea00640 1 -- 172.21.15.3:0/880510623 learned_addr learned my addr 172.21.15.3:0/880510623 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:16.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.657+0000 7f495f400640 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a9bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.658+0000 7f495ea00640 1 -- 172.21.15.3:0/880510623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 msgr2=0x561525a99190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.658+0000 7f495ea00640 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a99190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.658+0000 7f495ea00640 1 -- 172.21.15.3:0/880510623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 msgr2=0x561525a9bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.658+0000 7f495ea00640 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a9bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.658+0000 7f495ea00640 1 -- 172.21.15.3:0/880510623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615259927f0 con 0x561525a8e4f0 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.659+0000 7f495ea00640 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 0x561525a8e8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49500987f0 tx=0x7f495009ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc0554bbb3e9d276 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.659+0000 7f494fe00640 1 -- 172.21.15.3:0/880510623 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49500a01d0 con 0x561525a8e4f0 2024-08-01T04:23:16.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.660+0000 7f494fe00640 1 -- 172.21.15.3:0/880510623 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49500a4a90 con 0x561525a8e4f0 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.660+0000 7f494fe00640 1 -- 172.21.15.3:0/880510623 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495009b5a0 con 0x561525a8e4f0 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.661+0000 7f4968522740 1 -- 172.21.15.3:0/880510623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 msgr2=0x561525a8e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.661+0000 7f4968522740 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 0x561525a8e8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49500987f0 tx=0x7f495009ba20 comp rx=0 tx=0).stop 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 -- 172.21.15.3:0/880510623 shutdown_connections 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a9bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a99190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 --2- 172.21.15.3:0/880510623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8e4f0 0x561525a8e8d0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49500987f0 tx=0x7f495009ba20 comp rx=0 tx=0).stop 2024-08-01T04:23:16.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 -- 172.21.15.3:0/880510623 >> 172.21.15.3:0/880510623 conn(0x561525a8aa80 msgr2=0x561525a8ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 -- 172.21.15.3:0/880510623 shutdown_connections 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.662+0000 7f4968522740 1 -- 172.21.15.3:0/880510623 wait complete. 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.663+0000 7f4968522740 1 -- start start 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.663+0000 7f4968522740 1 Processor -- start 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.665+0000 7f4968522740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.666+0000 7f4968522740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.666+0000 7f4968522740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a973a0 0x561525aadf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.667+0000 7f495f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a973a0 0x561525aadf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.667+0000 7f495f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a973a0 0x561525aadf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:44322/0 (socket says 172.21.15.3:44322) 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.667+0000 7f495f400640 1 -- 172.21.15.3:0/106397909 learned_addr learned my addr 172.21.15.3:0/106397909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.668+0000 7f495ea00640 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.667+0000 7f4968522740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561525a9c2a0 con 0x561525a996d0 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.668+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5615259aecf0 con 0x561525a8ee10 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.668+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561525a9c420 con 0x561525a973a0 2024-08-01T04:23:16.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.668+0000 7f4968522740 10 client.0 _ll_register_callbacks cb 0x56152599ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.669+0000 7f495ea00640 1 -- 172.21.15.3:0/106397909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a973a0 msgr2=0x561525aadf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.669+0000 7f495ea00640 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a973a0 0x561525aadf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.669+0000 7f495ea00640 1 -- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 msgr2=0x561525a96e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.669+0000 7f495ea00640 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.669+0000 7f495ea00640 1 -- 172.21.15.3:0/106397909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615257653c0 con 0x561525a8ee10 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.670+0000 7f495ea00640 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a968d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f49500bc000 tx=0x7f49500986c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5853adf05a6df07 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.671+0000 7f494cc00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:23:16.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.671+0000 7f494cc00640 1 -- 172.21.15.3:0/106397909 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49500ccff0 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.671+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49500a4a90 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.671+0000 7f494cc00640 1 -- 172.21.15.3:0/106397909 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8673+0+0 (secure 0 0 0) 0x7f495009b4f0 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.672+0000 7f494cc00640 1 -- 172.21.15.3:0/106397909 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f49500a01d0 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.672+0000 7f4968522740 10 client.26063 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.673+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561525a9c2a0 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.673+0000 7f494cc00640 1 -- 172.21.15.3:0/106397909 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49500a4a90 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f494cc00640 1 -- 172.21.15.3:0/106397909 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f49500a1360 con 0x561525a8ee10 2024-08-01T04:23:16.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f4968522740 10 client.26063 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f4968522740 10 client.26063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f4968522740 20 client.26063 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f4968522740 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525ab1fe0 0x561525ab43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f4968522740 4 client.26063 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.675+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561525a01160 con 0x561525ab1fe0 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.677+0000 7f495e000640 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525ab1fe0 0x561525ab43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.678+0000 7f495e000640 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525ab1fe0 0x561525ab43d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81c55ca101757a32 server_cookie=ecb6c965f9493 in_seq=0 out_seq=0 2024-08-01T04:23:16.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.678+0000 7f494cc00640 10 client.26063 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.680+0000 7f494cc00640 1 -- 172.21.15.3:0/106397909 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561525a01160 con 0x561525ab1fe0 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.680+0000 7f494cc00640 10 client.26063 handle_command_reply: tid=0 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.680+0000 7f4968522740 1 client.26063 shutdown 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 msgr2=0x561525a968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a968d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f49500bc000 tx=0x7f49500986c0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 shutdown_connections 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525ab1fe0 0x561525ab43d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a973a0 0x561525aadf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a996d0 0x561525a96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 --2- 172.21.15.3:0/106397909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a8ee10 0x561525a968d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 >> 172.21.15.3:0/106397909 conn(0x561525a8aa80 msgr2=0x561525a907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.681+0000 7f494cc00640 0 client.26063 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.682+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 shutdown_connections 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.683+0000 7f4968522740 1 -- 172.21.15.3:0/106397909 wait complete. 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.683+0000 7f4968522740 20 client.26063 trim_cache size 0 max 16384 2024-08-01T04:23:16.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 Processor -- start 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 -- start start 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525aadf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525a90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525a951a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49601b6830 con 0x561525a973a0 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f49601b4100 con 0x561525a996d0 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.722+0000 7f4968522740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f49601b1f50 con 0x561525a8ee10 2024-08-01T04:23:16.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.724+0000 7f495f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525a951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.724+0000 7f495ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525aadf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.724+0000 7f495e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525a90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.724+0000 7f495ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525aadf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:44334/0 (socket says 172.21.15.3:44334) 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.725+0000 7f495ea00640 1 -- 172.21.15.3:0/3243505469 learned_addr learned my addr 172.21.15.3:0/3243505469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.726+0000 7f495ea00640 1 -- 172.21.15.3:0/3243505469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 msgr2=0x561525a951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.726+0000 7f495ea00640 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525a951a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.726+0000 7f495ea00640 1 -- 172.21.15.3:0/3243505469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 msgr2=0x561525a90f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.726+0000 7f495ea00640 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525a90f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.727+0000 7f495ea00640 1 -- 172.21.15.3:0/3243505469 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561525a94e20 con 0x561525a8ee10 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.729+0000 7f495ea00640 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525aadf60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f495009bb20 tx=0x7f495009ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6792d0a41162ce6 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.729+0000 7f494f400640 1 -- 172.21.15.3:0/3243505469 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49500056b0 con 0x561525a8ee10 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.729+0000 7f494f400640 1 -- 172.21.15.3:0/3243505469 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49500a4a90 con 0x561525a8ee10 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.729+0000 7f494f400640 1 -- 172.21.15.3:0/3243505469 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950097ef0 con 0x561525a8ee10 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.731+0000 7f4968522740 1 -- 172.21.15.3:0/3243505469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 msgr2=0x561525aadf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.731+0000 7f4968522740 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525aadf60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f495009bb20 tx=0x7f495009ba20 comp rx=0 tx=0).stop 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.732+0000 7f4968522740 1 -- 172.21.15.3:0/3243505469 shutdown_connections 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.732+0000 7f4968522740 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525a951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.732+0000 7f4968522740 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525a90f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.732+0000 7f4968522740 1 --2- 172.21.15.3:0/3243505469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525aadf60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.732+0000 7f4968522740 1 -- 172.21.15.3:0/3243505469 >> 172.21.15.3:0/3243505469 conn(0x561525a8aa80 msgr2=0x561525a91af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.733+0000 7f4968522740 1 -- 172.21.15.3:0/3243505469 shutdown_connections 2024-08-01T04:23:16.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.733+0000 7f4968522740 1 -- 172.21.15.3:0/3243505469 wait complete. 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.733+0000 7f4968522740 1 -- start start 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.734+0000 7f4968522740 1 Processor -- start 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.736+0000 7f4968522740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525ab43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.736+0000 7f4968522740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525ab4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f4968522740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525ab4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f4968522740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49601b25c0 con 0x561525a973a0 2024-08-01T04:23:16.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f4968522740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f49601b3a30 con 0x561525a996d0 2024-08-01T04:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f4968522740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f49601b2c90 con 0x561525a8ee10 2024-08-01T04:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f4968522740 10 client.0 _ll_register_callbacks cb 0x7f4950004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f495e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525ab4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f495ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525ab43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f495ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525ab43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:44336/0 (socket says 172.21.15.3:44336) 2024-08-01T04:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f495ea00640 1 -- 172.21.15.3:0/3459633798 learned_addr learned my addr 172.21.15.3:0/3459633798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.738+0000 7f495e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525ab4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39206/0 (socket says 172.21.15.3:39206) 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.739+0000 7f495ea00640 1 -- 172.21.15.3:0/3459633798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 msgr2=0x561525ab4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.739+0000 7f495ea00640 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525ab4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.739+0000 7f495ea00640 1 -- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 msgr2=0x561525ab4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.739+0000 7f495ea00640 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525ab4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.739+0000 7f495ea00640 1 -- 172.21.15.3:0/3459633798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561525b1aab0 con 0x561525a8ee10 2024-08-01T04:23:16.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.739+0000 7f495e000640 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525ab4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.741+0000 7f494cc00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.741+0000 7f495ea00640 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525ab43d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f49500058c0 tx=0x7f495009c5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b54d3bcaee10473 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.743+0000 7f494cc00640 1 -- 172.21.15.3:0/3459633798 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950097ed0 con 0x561525a8ee10 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.743+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49500a4a90 con 0x561525a8ee10 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.743+0000 7f494cc00640 1 -- 172.21.15.3:0/3459633798 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8673+0+0 (secure 0 0 0) 0x7f4950005000 con 0x561525a8ee10 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.744+0000 7f494cc00640 1 -- 172.21.15.3:0/3459633798 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4950085120 con 0x561525a8ee10 2024-08-01T04:23:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.744+0000 7f4968522740 10 client.26056 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.744+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561525b1aab0 con 0x561525a8ee10 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f494cc00640 1 -- 172.21.15.3:0/3459633798 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49500a4a90 con 0x561525a8ee10 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f494cc00640 1 -- 172.21.15.3:0/3459633798 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f49500a4ce0 con 0x561525a8ee10 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f4968522740 10 client.26056 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f4968522740 10 client.26056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f4968522740 20 client.26056 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f4968522740 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525b1c020 0x561525b1e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f4968522740 4 client.26056 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:23:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.746+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615259fb670 con 0x561525b1c020 2024-08-01T04:23:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.748+0000 7f495e000640 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525b1c020 0x561525b1e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.756+0000 7f495e000640 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525b1c020 0x561525b1e410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdf6195d5822042a server_cookie=82dd93153652534 in_seq=0 out_seq=0 2024-08-01T04:23:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.758+0000 7f494cc00640 10 client.26056 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:23:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.760+0000 7f494cc00640 1 -- 172.21.15.3:0/3459633798 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615259fb670 con 0x561525b1c020 2024-08-01T04:23:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.760+0000 7f494cc00640 10 client.26056 handle_command_reply: tid=0 2024-08-01T04:23:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.760+0000 7f4968522740 1 client.26056 shutdown 2024-08-01T04:23:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.762+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 msgr2=0x561525ab43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.762+0000 7f4968522740 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525ab43d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f49500058c0 tx=0x7f495009c5a0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.762+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 shutdown_connections 2024-08-01T04:23:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.762+0000 7f4968522740 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561525b1c020 0x561525b1e410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.762+0000 7f4968522740 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561525a996d0 0x561525ab4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.763+0000 7f4968522740 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561525a973a0 0x561525ab4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.763+0000 7f4968522740 1 --2- 172.21.15.3:0/3459633798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561525a8ee10 0x561525ab43d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.763+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 >> 172.21.15.3:0/3459633798 conn(0x561525a8aa80 msgr2=0x561525a95160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.763+0000 7f494cc00640 0 client.26056 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.763+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 shutdown_connections 2024-08-01T04:23:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.764+0000 7f4968522740 1 -- 172.21.15.3:0/3459633798 wait complete. 2024-08-01T04:23:16.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.766+0000 7f4968522740 20 client.26056 trim_cache size 0 max 16384 2024-08-01T04:23:16.398 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:23:16.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49440a2db0 msgr2=0x7f49440a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49440a2db0 0x7f49440a5270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f49480057c0 tx=0x7f494800b040 comp rx=0 tx=0).stop 2024-08-01T04:23:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 msgr2=0x7f49601a8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 0x7f49601a8640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9c0 tx=0x7f495000ee80 comp rx=0 tx=0).stop 2024-08-01T04:23:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 shutdown_connections 2024-08-01T04:23:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4960116be0 0x7f49601a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49440a2db0 0x7f49440a5270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4960109840 0x7f49601a8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 --2- 172.21.15.3:0/24927191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4960108f20 0x7f49601a8640 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 >> 172.21.15.3:0/24927191 conn(0x7f496006e000 msgr2=0x7f4960114690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 shutdown_connections 2024-08-01T04:23:16.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:15.770+0000 7f4965e00640 1 -- 172.21.15.3:0/24927191 wait complete. 2024-08-01T04:23:16.400 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:15 smithi003 ceph-mon[32262]: pgmap v213: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 66 MiB/s wr, 9.43k op/s 2024-08-01T04:23:17.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:23:17.464 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:23:18.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:17 smithi163 ceph-mon[37383]: pgmap v214: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 MiB/s wr, 7.26k op/s 2024-08-01T04:23:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:17 smithi003 ceph-mon[32262]: pgmap v214: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 MiB/s wr, 7.26k op/s 2024-08-01T04:23:18.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:17 smithi160 ceph-mon[37224]: pgmap v214: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 53 MiB/s wr, 7.26k op/s 2024-08-01T04:23:20.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:19 smithi003 ceph-mon[32262]: pgmap v215: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 54 MiB/s wr, 7.33k op/s 2024-08-01T04:23:20.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:19 smithi163 ceph-mon[37383]: pgmap v215: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 54 MiB/s wr, 7.33k op/s 2024-08-01T04:23:20.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:19 smithi160 ceph-mon[37224]: pgmap v215: 97 pgs: 97 active+clean; 25 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 54 MiB/s wr, 7.33k op/s 2024-08-01T04:23:22.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:21 smithi163 ceph-mon[37383]: pgmap v216: 97 pgs: 97 active+clean; 25 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 62 MiB/s wr, 8.73k op/s 2024-08-01T04:23:22.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:21 smithi003 ceph-mon[32262]: pgmap v216: 97 pgs: 97 active+clean; 25 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 62 MiB/s wr, 8.73k op/s 2024-08-01T04:23:22.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:21 smithi160 ceph-mon[37224]: pgmap v216: 97 pgs: 97 active+clean; 25 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 62 MiB/s wr, 8.73k op/s 2024-08-01T04:23:22.801 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:23:23.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.178+0000 7f3a82c00640 1 -- 172.21.15.3:0/4202455527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c102250 msgr2=0x7f3a7c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.178+0000 7f3a82c00640 1 --2- 172.21.15.3:0/4202455527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c102250 0x7f3a7c10e780 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009c10 tx=0x7f3a7402a6e0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.180+0000 7f3a82c00640 1 -- 172.21.15.3:0/4202455527 shutdown_connections 2024-08-01T04:23:23.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.180+0000 7f3a82c00640 1 --2- 172.21.15.3:0/4202455527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c102250 0x7f3a7c10e780 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.180+0000 7f3a82c00640 1 --2- 172.21.15.3:0/4202455527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a7c1018b0 0x7f3a7c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.180+0000 7f3a82c00640 1 --2- 172.21.15.3:0/4202455527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.180+0000 7f3a82c00640 1 -- 172.21.15.3:0/4202455527 >> 172.21.15.3:0/4202455527 conn(0x7f3a7c0fd5f0 msgr2=0x7f3a7c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.182+0000 7f3a82c00640 1 -- 172.21.15.3:0/4202455527 shutdown_connections 2024-08-01T04:23:23.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.182+0000 7f3a82c00640 1 -- 172.21.15.3:0/4202455527 wait complete. 2024-08-01T04:23:23.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.186+0000 7f3a82c00640 1 Processor -- start 2024-08-01T04:23:23.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.186+0000 7f3a82c00640 1 -- start start 2024-08-01T04:23:23.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.187+0000 7f3a82c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c1018b0 0x7f3a7c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.187+0000 7f3a82c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a7c102250 0x7f3a7c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.187+0000 7f3a82c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c10b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.187+0000 7f3a82c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c111cb0 con 0x7f3a7c1078b0 2024-08-01T04:23:23.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.187+0000 7f3a82c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c111b30 con 0x7f3a7c102250 2024-08-01T04:23:23.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.187+0000 7f3a82c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c111e30 con 0x7f3a7c1018b0 2024-08-01T04:23:23.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.188+0000 7f3a7b400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a7c102250 0x7f3a7c19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.188+0000 7f3a80c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c10b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.188+0000 7f3a80c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c10b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33770/0 (socket says 172.21.15.3:33770) 2024-08-01T04:23:23.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.188+0000 7f3a80c00640 1 -- 172.21.15.3:0/440564480 learned_addr learned my addr 172.21.15.3:0/440564480 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:23.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.189+0000 7f3a80c00640 1 -- 172.21.15.3:0/440564480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c1018b0 msgr2=0x7f3a7c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.189+0000 7f3a80c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c1018b0 0x7f3a7c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.189+0000 7f3a80c00640 1 -- 172.21.15.3:0/440564480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a7c102250 msgr2=0x7f3a7c19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.189+0000 7f3a80c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a7c102250 0x7f3a7c19f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.189+0000 7f3a80c00640 1 -- 172.21.15.3:0/440564480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c10c0f0 con 0x7f3a7c1078b0 2024-08-01T04:23:23.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.190+0000 7f3a80c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c10b830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a7402ac00 tx=0x7f3a74030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.191+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74041020 con 0x7f3a7c1078b0 2024-08-01T04:23:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.191+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a74003970 con 0x7f3a7c1078b0 2024-08-01T04:23:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.191+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74036040 con 0x7f3a7c1078b0 2024-08-01T04:23:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.192+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c10c380 con 0x7f3a7c1078b0 2024-08-01T04:23:23.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.192+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c19a9c0 con 0x7f3a7c1078b0 2024-08-01T04:23:23.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.194+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c102ff0 con 0x7f3a7c1078b0 2024-08-01T04:23:23.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.198+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f3a74044020 con 0x7f3a7c1078b0 2024-08-01T04:23:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.200+0000 7f3a78c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3a600a2d70 0x7f3a600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.201+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8673+0+0 (secure 0 0 0) 0x7f3a740ef7f0 con 0x7f3a7c1078b0 2024-08-01T04:23:23.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.203+0000 7f3a7be00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3a600a2d70 0x7f3a600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.205+0000 7f3a7be00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3a600a2d70 0x7f3a600a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c005fd0 tx=0x7f3a6c009600 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.212+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3a7404b020 con 0x7f3a7c1078b0 2024-08-01T04:23:23.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.670+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a7c101d10 con 0x7f3a7c1078b0 2024-08-01T04:23:23.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.672+0000 7f3a78c00640 1 -- 172.21.15.3:0/440564480 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f3a7c101d10 con 0x7f3a7c1078b0 2024-08-01T04:23:23.677 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:23:23.677 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:23:23.678 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:23:23.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3a600a2d70 msgr2=0x7f3a600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3a600a2d70 0x7f3a600a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c005fd0 tx=0x7f3a6c009600 comp rx=0 tx=0).stop 2024-08-01T04:23:23.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 msgr2=0x7f3a7c10b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c10b830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a7402ac00 tx=0x7f3a74030c60 comp rx=0 tx=0).stop 2024-08-01T04:23:23.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 shutdown_connections 2024-08-01T04:23:23.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a7c1078b0 0x7f3a7c10b830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a7c102250 0x7f3a7c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3a600a2d70 0x7f3a600a5230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 --2- 172.21.15.3:0/440564480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c1018b0 0x7f3a7c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.677+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 >> 172.21.15.3:0/440564480 conn(0x7f3a7c0fd5f0 msgr2=0x7f3a7c10d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.678+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 shutdown_connections 2024-08-01T04:23:23.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.678+0000 7f3a82c00640 1 -- 172.21.15.3:0/440564480 wait complete. 2024-08-01T04:23:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 -- 172.21.15.3:0/166487345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e010e2b0 msgr2=0x7f70e010e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 --2- 172.21.15.3:0/166487345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e010e2b0 0x7f70e010e690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009cd0 tx=0x7f70d002a8f0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 -- 172.21.15.3:0/166487345 shutdown_connections 2024-08-01T04:23:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 --2- 172.21.15.3:0/166487345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010d6e0 0x7f70e0115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 --2- 172.21.15.3:0/166487345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010ec60 0x7f70e010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 --2- 172.21.15.3:0/166487345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e010e2b0 0x7f70e010e690 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009cd0 tx=0x7f70d002a8f0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.710+0000 7f70e6400640 1 -- 172.21.15.3:0/166487345 >> 172.21.15.3:0/166487345 conn(0x7f70e006de70 msgr2=0x7f70e006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.714+0000 7f70e6400640 1 -- 172.21.15.3:0/166487345 shutdown_connections 2024-08-01T04:23:23.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.714+0000 7f70e6400640 1 -- 172.21.15.3:0/166487345 wait complete. 2024-08-01T04:23:23.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.715+0000 7f70e6400640 1 Processor -- start 2024-08-01T04:23:23.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 -- start start 2024-08-01T04:23:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 0x7f70e01a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010ec60 0x7f70e01a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e01a09c0 0x7f70e01a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70e011a4e0 con 0x7f70e010d6e0 2024-08-01T04:23:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f70e011a360 con 0x7f70e01a09c0 2024-08-01T04:23:23.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.717+0000 7f70e6400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70e011a660 con 0x7f70e010ec60 2024-08-01T04:23:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.719+0000 7f70df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 0x7f70e01a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.719+0000 7f70df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 0x7f70e01a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33792/0 (socket says 172.21.15.3:33792) 2024-08-01T04:23:23.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.719+0000 7f70df400640 1 -- 172.21.15.3:0/3008614727 learned_addr learned my addr 172.21.15.3:0/3008614727 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.719+0000 7f70dfe00640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e01a09c0 0x7f70e01a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.720+0000 7f70dea00640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010ec60 0x7f70e01a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.720+0000 7f70df400640 1 -- 172.21.15.3:0/3008614727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010ec60 msgr2=0x7f70e01a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.720+0000 7f70df400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010ec60 0x7f70e01a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.720+0000 7f70df400640 1 -- 172.21.15.3:0/3008614727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e01a09c0 msgr2=0x7f70e01a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.720+0000 7f70df400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e01a09c0 0x7f70e01a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.720+0000 7f70df400640 1 -- 172.21.15.3:0/3008614727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e01a13b0 con 0x7f70e010d6e0 2024-08-01T04:23:23.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.722+0000 7f70dea00640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010ec60 0x7f70e01a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:23:23.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.722+0000 7f70df400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 0x7f70e01a6800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009430 tx=0x7f70d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.724+0000 7f70d7e00640 1 -- 172.21.15.3:0/3008614727 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0030b40 con 0x7f70e010d6e0 2024-08-01T04:23:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.725+0000 7f70d7e00640 1 -- 172.21.15.3:0/3008614727 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d0040030 con 0x7f70e010d6e0 2024-08-01T04:23:23.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.725+0000 7f70d7e00640 1 -- 172.21.15.3:0/3008614727 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d00045a0 con 0x7f70e010d6e0 2024-08-01T04:23:23.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.727+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e01a1640 con 0x7f70e010d6e0 2024-08-01T04:23:23.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.728+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e01ab890 con 0x7f70e010d6e0 2024-08-01T04:23:23.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.735+0000 7f70d7e00640 1 -- 172.21.15.3:0/3008614727 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f70d0030ce0 con 0x7f70e010d6e0 2024-08-01T04:23:23.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.738+0000 7f70d7e00640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70c40a2e40 0x7f70c40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.739+0000 7f70d7e00640 1 -- 172.21.15.3:0/3008614727 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f70d00f0180 con 0x7f70e010d6e0 2024-08-01T04:23:23.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.739+0000 7f70dea00640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70c40a2e40 0x7f70c40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.742+0000 7f70dea00640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70c40a2e40 0x7f70c40a5300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f70e0073190 tx=0x7f70cc00a4c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.756+0000 7f70e8e28740 1 Processor -- start 2024-08-01T04:23:23.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.761+0000 7f70e8e28740 1 -- start start 2024-08-01T04:23:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.761+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.765+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.765+0000 7f70df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.766+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e70ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.766+0000 7f70e8e28740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b4f6dc8bf0 con 0x55b4f6e634f0 2024-08-01T04:23:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.766+0000 7f70e8e28740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b4f6d60b20 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.766+0000 7f70e8e28740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b4f6dc4bb0 con 0x55b4f6e63e10 2024-08-01T04:23:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.767+0000 7f70dea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.768+0000 7f70df400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e638d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33804/0 (socket says 172.21.15.3:33804) 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.768+0000 7f70df400640 1 -- 172.21.15.3:0/1692158648 learned_addr learned my addr 172.21.15.3:0/1692158648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.768+0000 7f70dfe00640 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e70ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.769+0000 7f70dfe00640 1 -- 172.21.15.3:0/1692158648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 msgr2=0x55b4f6e6e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.769+0000 7f70dfe00640 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.769+0000 7f70dfe00640 1 -- 172.21.15.3:0/1692158648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 msgr2=0x55b4f6e638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.769+0000 7f70dfe00640 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e638d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.769+0000 7f70dfe00640 1 -- 172.21.15.3:0/1692158648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f6d677f0 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.770+0000 7f70dfe00640 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e70ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f70d800ab50 tx=0x7f70d800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c73edeb6c71016 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.770+0000 7f70d4c00640 1 -- 172.21.15.3:0/1692158648 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8005220 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.770+0000 7f70d4c00640 1 -- 172.21.15.3:0/1692158648 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d800ae50 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.770+0000 7f70d4c00640 1 -- 172.21.15.3:0/1692158648 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d80188d0 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 -- 172.21.15.3:0/1692158648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 msgr2=0x55b4f6e70ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e70ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f70d800ab50 tx=0x7f70d800f790 comp rx=0 tx=0).stop 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 -- 172.21.15.3:0/1692158648 shutdown_connections 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e70ac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1692158648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.771+0000 7f70e8e28740 1 -- 172.21.15.3:0/1692158648 >> 172.21.15.3:0/1692158648 conn(0x55b4f6e5fa80 msgr2=0x55b4f6e5fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.773+0000 7f70e8e28740 1 -- 172.21.15.3:0/1692158648 shutdown_connections 2024-08-01T04:23:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.773+0000 7f70e8e28740 1 -- 172.21.15.3:0/1692158648 wait complete. 2024-08-01T04:23:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.774+0000 7f70e8e28740 1 -- start start 2024-08-01T04:23:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.774+0000 7f70e8e28740 1 Processor -- start 2024-08-01T04:23:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e6c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b4f6e712a0 con 0x55b4f6e634f0 2024-08-01T04:23:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b4f6d83cf0 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b4f6e71420 con 0x55b4f6e63e10 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.775+0000 7f70e8e28740 10 client.0 _ll_register_callbacks cb 0x55b4f6d6fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.776+0000 7f70dea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.776+0000 7f70dea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51138/0 (socket says 172.21.15.3:51138) 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.776+0000 7f70dfe00640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e6c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.776+0000 7f70dea00640 1 -- 172.21.15.3:0/3648011142 learned_addr learned my addr 172.21.15.3:0/3648011142 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.776+0000 7f70df400640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.777+0000 7f70dfe00640 1 -- 172.21.15.3:0/3648011142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 msgr2=0x55b4f6e6baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.777+0000 7f70dfe00640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.777+0000 7f70dfe00640 1 -- 172.21.15.3:0/3648011142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 msgr2=0x55b4f6e6b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.777+0000 7f70dfe00640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.777+0000 7f70dfe00640 1 -- 172.21.15.3:0/3648011142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f6b3a3c0 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.778+0000 7f70c9600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:23:23.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.779+0000 7f70dfe00640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e6c030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70d800ab20 tx=0x7f70d8018880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1db367d3678656 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.779+0000 7f70c9600640 1 -- 172.21.15.3:0/3648011142 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8027020 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.779+0000 7f70c9600640 1 -- 172.21.15.3:0/3648011142 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f70d801e040 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.779+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d800ae50 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.780+0000 7f70c9600640 1 -- 172.21.15.3:0/3648011142 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70d8027020 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.780+0000 7f70e8e28740 10 client.26081 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:23.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.780+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4f6e8df60 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.782+0000 7f70c9600640 1 -- 172.21.15.3:0/3648011142 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d8018c80 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.782+0000 7f70c9600640 1 -- 172.21.15.3:0/3648011142 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f70d8023470 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.783+0000 7f70e8e28740 10 client.26081 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:23.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.783+0000 7f70e8e28740 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:23:23.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.783+0000 7f70e8e28740 20 client.26081 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:23.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.783+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6e91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.784+0000 7f70e8e28740 4 client.26081 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:23:23.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.784+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f70d800ae50 con 0x55b4f6e8f310 2024-08-01T04:23:23.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.784+0000 7f70df400640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6e91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.786+0000 7f70c9600640 10 client.26081 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:23.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.786+0000 7f70df400640 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6e91700 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a1fb30e461424 server_cookie=58ec3b8416bf21f7 in_seq=0 out_seq=0 2024-08-01T04:23:23.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.787+0000 7f70c9600640 1 -- 172.21.15.3:0/3648011142 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f70d800ae50 con 0x55b4f6e8f310 2024-08-01T04:23:23.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.787+0000 7f70c9600640 10 client.26081 handle_command_reply: tid=0 2024-08-01T04:23:23.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.787+0000 7f70e8e28740 1 client.26081 shutdown 2024-08-01T04:23:23.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.788+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 msgr2=0x55b4f6e6c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.788+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e6c030 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70d800ab20 tx=0x7f70d8018880 comp rx=0 tx=0).stop 2024-08-01T04:23:23.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.788+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 shutdown_connections 2024-08-01T04:23:23.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.788+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6e91700 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.788+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e6c030 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.789+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.788+0000 7f70c9600640 0 client.26081 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:23.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.789+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3648011142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.789+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 >> 172.21.15.3:0/3648011142 conn(0x55b4f6e5fa80 msgr2=0x55b4f6e657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.790+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 shutdown_connections 2024-08-01T04:23:23.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.790+0000 7f70e8e28740 1 -- 172.21.15.3:0/3648011142 wait complete. 2024-08-01T04:23:23.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.792+0000 7f70e8e28740 20 client.26081 trim_cache size 0 max 16384 2024-08-01T04:23:23.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.825+0000 7f70e8e28740 1 Processor -- start 2024-08-01T04:23:23.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.825+0000 7f70e8e28740 1 -- start start 2024-08-01T04:23:23.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.826+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.826+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.826+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e96d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.827+0000 7f70dea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.827+0000 7f70dea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:35512/0 (socket says 172.21.15.3:35512) 2024-08-01T04:23:23.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.827+0000 7f70dea00640 1 -- 172.21.15.3:0/3744151917 learned_addr learned my addr 172.21.15.3:0/3744151917 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:23.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.827+0000 7f70df400640 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.827+0000 7f70dfe00640 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e96d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.828+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01b6d00 con 0x55b4f6e634f0 2024-08-01T04:23:23.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.828+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01b45d0 con 0x55b4f6e63e10 2024-08-01T04:23:23.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.828+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01b2440 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.830+0000 7f70dea00640 1 -- 172.21.15.3:0/3744151917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e6e6d0 msgr2=0x55b4f6e96d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.830+0000 7f70dea00640 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e96d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.830+0000 7f70dea00640 1 -- 172.21.15.3:0/3744151917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 msgr2=0x55b4f6e6bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.830+0000 7f70dea00640 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.830+0000 7f70dea00640 1 -- 172.21.15.3:0/3744151917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f6e6e100 con 0x55b4f6e63e10 2024-08-01T04:23:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.832+0000 7f70dea00640 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6d520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005c80 tx=0x7f70cc011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fdc56da76234cfe server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.832+0000 7f70cbe00640 1 -- 172.21.15.3:0/3744151917 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc016ba0 con 0x55b4f6e63e10 2024-08-01T04:23:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.833+0000 7f70cbe00640 1 -- 172.21.15.3:0/3744151917 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc027070 con 0x55b4f6e63e10 2024-08-01T04:23:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.833+0000 7f70cbe00640 1 -- 172.21.15.3:0/3744151917 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc015600 con 0x55b4f6e63e10 2024-08-01T04:23:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.833+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 msgr2=0x55b4f6e6d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.833+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6d520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005c80 tx=0x7f70cc011000 comp rx=0 tx=0).stop 2024-08-01T04:23:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.835+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 shutdown_connections 2024-08-01T04:23:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.835+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e96d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.835+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e6d520 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.835+0000 7f70e8e28740 1 --2- 172.21.15.3:0/3744151917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e634f0 0x55b4f6e6bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.835+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 >> 172.21.15.3:0/3744151917 conn(0x55b4f6e5fa80 msgr2=0x55b4f6e6ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.837+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 shutdown_connections 2024-08-01T04:23:23.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.837+0000 7f70e8e28740 1 -- 172.21.15.3:0/3744151917 wait complete. 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.838+0000 7f70e8e28740 1 -- start start 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.838+0000 7f70e8e28740 1 Processor -- start 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.839+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e634f0 0x55b4f6e687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.839+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e68d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.839+0000 7f70e8e28740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e69270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.839+0000 7f70dea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e68d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.839+0000 7f70dfe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e69270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70e8e28740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01b2a90 con 0x55b4f6e6e6d0 2024-08-01T04:23:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70e8e28740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01b3f00 con 0x55b4f6e63e10 2024-08-01T04:23:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70e8e28740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f70e01b3160 con 0x55b4f6e634f0 2024-08-01T04:23:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70e8e28740 10 client.0 _ll_register_callbacks cb 0x7f70d80043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e68d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:35518/0 (socket says 172.21.15.3:35518) 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 -- 172.21.15.3:0/1164358867 learned_addr learned my addr 172.21.15.3:0/1164358867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 -- 172.21.15.3:0/1164358867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e634f0 msgr2=0x55b4f6e687f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e634f0 0x55b4f6e687f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 -- 172.21.15.3:0/1164358867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e6e6d0 msgr2=0x55b4f6e69270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e69270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.840+0000 7f70dea00640 1 -- 172.21.15.3:0/1164358867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f6ef8e90 con 0x55b4f6e63e10 2024-08-01T04:23:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.841+0000 7f70dfe00640 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e69270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:23:23.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.845+0000 7f70dea00640 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e68d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f70cc022e60 tx=0x7f70cc022620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f533ed4df3755033 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:23.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.845+0000 7f70c9600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:23:23.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.846+0000 7f70c9600640 1 -- 172.21.15.3:0/1164358867 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc02f070 con 0x55b4f6e63e10 2024-08-01T04:23:23.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.846+0000 7f70c9600640 1 -- 172.21.15.3:0/1164358867 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f70cc02b860 con 0x55b4f6e63e10 2024-08-01T04:23:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.847+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70cc027070 con 0x55b4f6e63e10 2024-08-01T04:23:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.848+0000 7f70c9600640 1 -- 172.21.15.3:0/1164358867 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70cc039050 con 0x55b4f6e63e10 2024-08-01T04:23:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.849+0000 7f70e8e28740 10 client.26087 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:23.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.849+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4f6efa170 con 0x55b4f6e63e10 2024-08-01T04:23:23.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.849+0000 7f70c9600640 1 -- 172.21.15.3:0/1164358867 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc027070 con 0x55b4f6e63e10 2024-08-01T04:23:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.850+0000 7f70c9600640 1 -- 172.21.15.3:0/1164358867 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f70cc034670 con 0x55b4f6e63e10 2024-08-01T04:23:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.850+0000 7f70e8e28740 10 client.26087 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:23.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.851+0000 7f70e8e28740 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:23:23.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.851+0000 7f70e8e28740 20 client.26087 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:23.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.851+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6efc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:23.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.851+0000 7f70e8e28740 4 client.26087 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:23:23.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.851+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4f6dd6160 con 0x55b4f6e8f310 2024-08-01T04:23:23.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.851+0000 7f70df400640 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6efc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.855+0000 7f70df400640 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6efc900 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198649549857cbc2 server_cookie=e8374304e748aec7 in_seq=0 out_seq=0 2024-08-01T04:23:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.855+0000 7f70c9600640 10 client.26087 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.855+0000 7f70c9600640 1 -- 172.21.15.3:0/1164358867 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4f6dd6160 con 0x55b4f6e8f310 2024-08-01T04:23:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.855+0000 7f70c9600640 10 client.26087 handle_command_reply: tid=0 2024-08-01T04:23:23.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.855+0000 7f70e8e28740 1 client.26087 shutdown 2024-08-01T04:23:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 msgr2=0x55b4f6e68d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e68d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f70cc022e60 tx=0x7f70cc022620 comp rx=0 tx=0).stop 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 shutdown_connections 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b4f6e8f310 0x55b4f6efc900 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4f6e6e6d0 0x55b4f6e69270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b4f6e63e10 0x55b4f6e68d30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 --2- 172.21.15.3:0/1164358867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b4f6e634f0 0x55b4f6e687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70c9600640 0 client.26087 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:23.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.856+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 >> 172.21.15.3:0/1164358867 conn(0x55b4f6e5fa80 msgr2=0x55b4f6e5ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.857+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 shutdown_connections 2024-08-01T04:23:23.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.857+0000 7f70e8e28740 1 -- 172.21.15.3:0/1164358867 wait complete. 2024-08-01T04:23:23.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.857+0000 7f70e8e28740 20 client.26087 trim_cache size 0 max 16384 2024-08-01T04:23:23.860 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:23:23.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.858+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70c40a2e40 msgr2=0x7f70c40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.858+0000 7f70e6400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70c40a2e40 0x7f70c40a5300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f70e0073190 tx=0x7f70cc00a4c0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.858+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 msgr2=0x7f70e01a6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:23.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.858+0000 7f70e6400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 0x7f70e01a6800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009430 tx=0x7f70d0003c10 comp rx=0 tx=0).stop 2024-08-01T04:23:23.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 shutdown_connections 2024-08-01T04:23:23.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e01a09c0 0x7f70e01a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70c40a2e40 0x7f70c40a5300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e010ec60 0x7f70e01a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 --2- 172.21.15.3:0/3008614727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e010d6e0 0x7f70e01a6800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:23.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 >> 172.21.15.3:0/3008614727 conn(0x7f70e006de70 msgr2=0x7f70e010b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:23.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 shutdown_connections 2024-08-01T04:23:23.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:23.859+0000 7f70e6400640 1 -- 172.21.15.3:0/3008614727 wait complete. 2024-08-01T04:23:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:23 smithi003 ceph-mon[32262]: pgmap v217: 97 pgs: 97 active+clean; 25 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 MiB/s wr, 8.55k op/s 2024-08-01T04:23:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:23 smithi003 ceph-mon[32262]: osdmap e93: 12 total, 12 up, 12 in 2024-08-01T04:23:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:23 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/440564480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:23:24.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:23 smithi163 ceph-mon[37383]: pgmap v217: 97 pgs: 97 active+clean; 25 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 MiB/s wr, 8.55k op/s 2024-08-01T04:23:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:23 smithi163 ceph-mon[37383]: osdmap e93: 12 total, 12 up, 12 in 2024-08-01T04:23:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:23 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/440564480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:23:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:23 smithi160 ceph-mon[37224]: pgmap v217: 97 pgs: 97 active+clean; 25 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 MiB/s wr, 8.55k op/s 2024-08-01T04:23:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:23 smithi160 ceph-mon[37224]: osdmap e93: 12 total, 12 up, 12 in 2024-08-01T04:23:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:23 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/440564480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:23:25.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:24 smithi163 ceph-mon[37383]: pgmap v219: 97 pgs: 97 active+clean; 25 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 63 MiB/s wr, 9.22k op/s 2024-08-01T04:23:25.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:24 smithi003 ceph-mon[32262]: pgmap v219: 97 pgs: 97 active+clean; 25 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 63 MiB/s wr, 9.22k op/s 2024-08-01T04:23:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:24 smithi160 ceph-mon[37224]: pgmap v219: 97 pgs: 97 active+clean; 25 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 63 MiB/s wr, 9.22k op/s 2024-08-01T04:23:27.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:27 smithi163 ceph-mon[37383]: pgmap v220: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 76 MiB/s wr, 11.56k op/s 2024-08-01T04:23:27.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:27 smithi003 ceph-mon[32262]: pgmap v220: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 76 MiB/s wr, 11.56k op/s 2024-08-01T04:23:27.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:27 smithi160 ceph-mon[37224]: pgmap v220: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 76 MiB/s wr, 11.56k op/s 2024-08-01T04:23:29.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:29 smithi163 ceph-mon[37383]: pgmap v221: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 27 KiB/s rd, 79 MiB/s wr, 12.05k op/s 2024-08-01T04:23:29.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:29 smithi003 ceph-mon[32262]: pgmap v221: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 27 KiB/s rd, 79 MiB/s wr, 12.05k op/s 2024-08-01T04:23:29.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:29 smithi160 ceph-mon[37224]: pgmap v221: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 27 KiB/s rd, 79 MiB/s wr, 12.05k op/s 2024-08-01T04:23:31.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:31 smithi163 ceph-mon[37383]: pgmap v222: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 7.72k op/s 2024-08-01T04:23:31.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:31 smithi003 ceph-mon[32262]: pgmap v222: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 7.72k op/s 2024-08-01T04:23:31.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:31 smithi160 ceph-mon[37224]: pgmap v222: 97 pgs: 97 active+clean; 26 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 62 KiB/s rd, 49 MiB/s wr, 7.72k op/s 2024-08-01T04:23:32.579 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24548 quiesce db --cancel --all' 2024-08-01T04:23:32.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- timeout 105 ceph tell mds.24548 quiesce db --cancel --all 2024-08-01T04:23:33.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:33 smithi163 ceph-mon[37383]: pgmap v223: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 62 KiB/s rd, 50 MiB/s wr, 8.09k op/s 2024-08-01T04:23:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:33 smithi003 ceph-mon[32262]: pgmap v223: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 62 KiB/s rd, 50 MiB/s wr, 8.09k op/s 2024-08-01T04:23:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:33 smithi160 ceph-mon[37224]: pgmap v223: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 62 KiB/s rd, 50 MiB/s wr, 8.09k op/s 2024-08-01T04:23:35.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:35 smithi163 ceph-mon[37383]: pgmap v224: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 59 KiB/s rd, 52 MiB/s wr, 8.33k op/s 2024-08-01T04:23:35.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:35 smithi003 ceph-mon[32262]: pgmap v224: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 59 KiB/s rd, 52 MiB/s wr, 8.33k op/s 2024-08-01T04:23:35.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:35 smithi160 ceph-mon[37224]: pgmap v224: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 59 KiB/s rd, 52 MiB/s wr, 8.33k op/s 2024-08-01T04:23:36.009 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:23:36.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:23:36.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:23:37.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:37 smithi163 ceph-mon[37383]: pgmap v225: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 68 KiB/s rd, 59 MiB/s wr, 9.33k op/s 2024-08-01T04:23:37.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:37 smithi003 ceph-mon[32262]: pgmap v225: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 68 KiB/s rd, 59 MiB/s wr, 9.33k op/s 2024-08-01T04:23:37.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:37 smithi160 ceph-mon[37224]: pgmap v225: 97 pgs: 97 active+clean; 26 GiB data, 50 GiB used, 1022 GiB / 1.0 TiB avail; 68 KiB/s rd, 59 MiB/s wr, 9.33k op/s 2024-08-01T04:23:39.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:39 smithi163 ceph-mon[37383]: pgmap v226: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 48 KiB/s rd, 45 MiB/s wr, 7.01k op/s 2024-08-01T04:23:39.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:39 smithi003 ceph-mon[32262]: pgmap v226: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 48 KiB/s rd, 45 MiB/s wr, 7.01k op/s 2024-08-01T04:23:39.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:39 smithi160 ceph-mon[37224]: pgmap v226: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 48 KiB/s rd, 45 MiB/s wr, 7.01k op/s 2024-08-01T04:23:41.239 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:41 smithi003 ceph-mon[32262]: pgmap v227: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 74 KiB/s rd, 49 MiB/s wr, 7.50k op/s 2024-08-01T04:23:41.276 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.3 2024-08-01T04:23:41.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:23:41.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:41 smithi163 ceph-mon[37383]: pgmap v227: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 74 KiB/s rd, 49 MiB/s wr, 7.50k op/s 2024-08-01T04:23:41.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:41 smithi160 ceph-mon[37224]: pgmap v227: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 74 KiB/s rd, 49 MiB/s wr, 7.50k op/s 2024-08-01T04:23:44.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:44 smithi160 ceph-mon[37224]: pgmap v228: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 46 KiB/s rd, 55 MiB/s wr, 8.21k op/s 2024-08-01T04:23:45.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:44 smithi163 ceph-mon[37383]: pgmap v228: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 46 KiB/s rd, 55 MiB/s wr, 8.21k op/s 2024-08-01T04:23:45.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:44 smithi003 ceph-mon[32262]: pgmap v228: 97 pgs: 97 active+clean; 26 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 46 KiB/s rd, 55 MiB/s wr, 8.21k op/s 2024-08-01T04:23:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:45 smithi160 ceph-mon[37224]: pgmap v229: 97 pgs: 97 active+clean; 26 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 46 KiB/s rd, 56 MiB/s wr, 8.28k op/s 2024-08-01T04:23:46.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:45 smithi163 ceph-mon[37383]: pgmap v229: 97 pgs: 97 active+clean; 26 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 46 KiB/s rd, 56 MiB/s wr, 8.28k op/s 2024-08-01T04:23:46.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:45 smithi003 ceph-mon[32262]: pgmap v229: 97 pgs: 97 active+clean; 26 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 46 KiB/s rd, 56 MiB/s wr, 8.28k op/s 2024-08-01T04:23:47.847 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:47 smithi163 ceph-mon[37383]: pgmap v230: 97 pgs: 97 active+clean; 27 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 67 KiB/s rd, 54 MiB/s wr, 8.00k op/s 2024-08-01T04:23:47.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:47 smithi160 ceph-mon[37224]: pgmap v230: 97 pgs: 97 active+clean; 27 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 67 KiB/s rd, 54 MiB/s wr, 8.00k op/s 2024-08-01T04:23:48.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:47 smithi003 ceph-mon[32262]: pgmap v230: 97 pgs: 97 active+clean; 27 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 67 KiB/s rd, 54 MiB/s wr, 8.00k op/s 2024-08-01T04:23:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:49 smithi160 ceph-mon[37224]: pgmap v231: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 6.49k op/s 2024-08-01T04:23:50.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:49 smithi163 ceph-mon[37383]: pgmap v231: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 6.49k op/s 2024-08-01T04:23:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:49 smithi003 ceph-mon[32262]: pgmap v231: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 6.49k op/s 2024-08-01T04:23:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:51 smithi160 ceph-mon[37224]: pgmap v232: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 61 KiB/s rd, 47 MiB/s wr, 7.14k op/s 2024-08-01T04:23:51.939 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:51 smithi003 ceph-mon[32262]: pgmap v232: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 61 KiB/s rd, 47 MiB/s wr, 7.14k op/s 2024-08-01T04:23:52.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:51 smithi163 ceph-mon[37383]: pgmap v232: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 61 KiB/s rd, 47 MiB/s wr, 7.14k op/s 2024-08-01T04:23:52.050 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:23:52.058 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 -- 172.21.15.3:0/948163389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980b7540 msgr2=0x7f49980b9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 --2- 172.21.15.3:0/948163389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980b7540 0x7f49980b9930 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49a0051230 tx=0x7f49a008e330 comp rx=0 tx=0).stop 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 -- 172.21.15.3:0/948163389 shutdown_connections 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 --2- 172.21.15.3:0/948163389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980b7540 0x7f49980b9930 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 --2- 172.21.15.3:0/948163389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980a5140 0x7f49980b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 --2- 172.21.15.3:0/948163389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980a4820 0x7f49980a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.009+0000 7f49a5000640 1 -- 172.21.15.3:0/948163389 >> 172.21.15.3:0/948163389 conn(0x7f499801a0e0 msgr2=0x7f499801a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.010+0000 7f49a5000640 1 -- 172.21.15.3:0/948163389 shutdown_connections 2024-08-01T04:23:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.010+0000 7f49a5000640 1 -- 172.21.15.3:0/948163389 wait complete. 2024-08-01T04:23:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 Processor -- start 2024-08-01T04:23:53.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 -- start start 2024-08-01T04:23:53.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980a4820 0x7f4998143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 0x7f49981438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980b7540 0x7f4998147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49980bcec0 con 0x7f49980a4820 2024-08-01T04:23:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f49980bcd40 con 0x7f49980a5140 2024-08-01T04:23:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.011+0000 7f49a5000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f49980bd040 con 0x7f49980b7540 2024-08-01T04:23:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.012+0000 7f499ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 0x7f49981438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.013+0000 7f499ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 0x7f49981438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:51984/0 (socket says 172.21.15.3:51984) 2024-08-01T04:23:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.013+0000 7f499ea00640 1 -- 172.21.15.3:0/2659511881 learned_addr learned my addr 172.21.15.3:0/2659511881 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.013+0000 7f499f400640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980a4820 0x7f4998143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.014+0000 7f499ea00640 1 -- 172.21.15.3:0/2659511881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980b7540 msgr2=0x7f4998147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.014+0000 7f499ea00640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980b7540 0x7f4998147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.014+0000 7f499ea00640 1 -- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980a4820 msgr2=0x7f4998143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.014+0000 7f499ea00640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980a4820 0x7f4998143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.014+0000 7f499ea00640 1 -- 172.21.15.3:0/2659511881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4998148330 con 0x7f49980a5140 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.016+0000 7f499ea00640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 0x7f49981438c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e9f0 tx=0x7f498c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.017+0000 7f4993e00640 1 -- 172.21.15.3:0/2659511881 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00ccc0 con 0x7f49980a5140 2024-08-01T04:23:53.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.017+0000 7f4993e00640 1 -- 172.21.15.3:0/2659511881 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c010910 con 0x7f49980a5140 2024-08-01T04:23:53.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.017+0000 7f4993e00640 1 -- 172.21.15.3:0/2659511881 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0188a0 con 0x7f49980a5140 2024-08-01T04:23:53.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.020+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4998148620 con 0x7f49980a5140 2024-08-01T04:23:53.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.020+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49980ac7e0 con 0x7f49980a5140 2024-08-01T04:23:53.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.028+0000 7f4993e00640 1 -- 172.21.15.3:0/2659511881 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f498c010ac0 con 0x7f49980a5140 2024-08-01T04:23:53.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.032+0000 7f4993e00640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49880a2ed0 0x7f49880a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.033+0000 7f4993e00640 1 -- 172.21.15.3:0/2659511881 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f498c0cf880 con 0x7f49980a5140 2024-08-01T04:23:53.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.033+0000 7f499f400640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49880a2ed0 0x7f49880a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.034+0000 7f499f400640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49880a2ed0 0x7f49880a5390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f499400abe0 tx=0x7f499400a2d0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:53.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.077+0000 7f49a90a8740 1 Processor -- start 2024-08-01T04:23:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.077+0000 7f49a90a8740 1 -- start start 2024-08-01T04:23:53.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.078+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f60b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.078+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.078+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f6dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.078+0000 7f49a90a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555f6ec5a70 con 0x5555f6f6b900 2024-08-01T04:23:53.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.078+0000 7f49a90a8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5555f6e5dab0 con 0x5555f6f61040 2024-08-01T04:23:53.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.078+0000 7f49a90a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5555f6ec1a30 con 0x5555f6f60720 2024-08-01T04:23:53.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.081+0000 7f499f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f60b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.081+0000 7f499f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f60b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53228/0 (socket says 172.21.15.3:53228) 2024-08-01T04:23:53.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.081+0000 7f499f400640 1 -- 172.21.15.3:0/2914263184 learned_addr learned my addr 172.21.15.3:0/2914263184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.081+0000 7f499fe00640 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f6dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.083+0000 7f499ea00640 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.085+0000 7f499f400640 1 -- 172.21.15.3:0/2914263184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 msgr2=0x5555f6f6b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.085+0000 7f499f400640 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.085+0000 7f499f400640 1 -- 172.21.15.3:0/2914263184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 msgr2=0x5555f6f6dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.085+0000 7f499f400640 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f6dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.085+0000 7f499f400640 1 -- 172.21.15.3:0/2914263184 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f6e64850 con 0x5555f6f60720 2024-08-01T04:23:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.086+0000 7f499f400640 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f60b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4994020fe0 tx=0x7f4994023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde6dce687cbb1f server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.087+0000 7f4990c00640 1 -- 172.21.15.3:0/2914263184 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994027070 con 0x5555f6f60720 2024-08-01T04:23:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.087+0000 7f4990c00640 1 -- 172.21.15.3:0/2914263184 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f499402c900 con 0x5555f6f60720 2024-08-01T04:23:53.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.087+0000 7f4990c00640 1 -- 172.21.15.3:0/2914263184 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499402da10 con 0x5555f6f60720 2024-08-01T04:23:53.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.089+0000 7f49a90a8740 1 -- 172.21.15.3:0/2914263184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 msgr2=0x5555f6f60b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.089+0000 7f49a90a8740 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f60b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4994020fe0 tx=0x7f4994023000 comp rx=0 tx=0).stop 2024-08-01T04:23:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.091+0000 7f49a90a8740 1 -- 172.21.15.3:0/2914263184 shutdown_connections 2024-08-01T04:23:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.091+0000 7f49a90a8740 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f6dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.091+0000 7f49a90a8740 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.091+0000 7f49a90a8740 1 --2- 172.21.15.3:0/2914263184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f60b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.091+0000 7f49a90a8740 1 -- 172.21.15.3:0/2914263184 >> 172.21.15.3:0/2914263184 conn(0x5555f6f5ccb0 msgr2=0x5555f6f5d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.093+0000 7f49a90a8740 1 -- 172.21.15.3:0/2914263184 shutdown_connections 2024-08-01T04:23:53.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.093+0000 7f49a90a8740 1 -- 172.21.15.3:0/2914263184 wait complete. 2024-08-01T04:23:53.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.094+0000 7f49a90a8740 1 -- start start 2024-08-01T04:23:53.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.095+0000 7f49a90a8740 1 Processor -- start 2024-08-01T04:23:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.098+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.099+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.099+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f69280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.099+0000 7f49a90a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555f6f6e4d0 con 0x5555f6f6b900 2024-08-01T04:23:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.099+0000 7f49a90a8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5555f6e80bc0 con 0x5555f6f61040 2024-08-01T04:23:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.099+0000 7f49a90a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5555f6f6e650 con 0x5555f6f60720 2024-08-01T04:23:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.100+0000 7f49a90a8740 10 client.0 _ll_register_callbacks cb 0x5555f6bd4b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:53.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.100+0000 7f499f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.100+0000 7f499ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.100+0000 7f499f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f68800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53230/0 (socket says 172.21.15.3:53230) 2024-08-01T04:23:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.101+0000 7f499f400640 1 -- 172.21.15.3:0/1757084326 learned_addr learned my addr 172.21.15.3:0/1757084326 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:53.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.101+0000 7f499fe00640 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f69280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.102+0000 7f499fe00640 1 -- 172.21.15.3:0/1757084326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 msgr2=0x5555f6f68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.102+0000 7f499fe00640 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f68800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.102+0000 7f499fe00640 1 -- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 msgr2=0x5555f6f68d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.102+0000 7f499fe00640 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f68d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.102+0000 7f499fe00640 1 -- 172.21.15.3:0/1757084326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f6c37620 con 0x5555f6f6b900 2024-08-01T04:23:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.105+0000 7f4985600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:23:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.105+0000 7f499fe00640 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f69280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49a0065770 tx=0x7f49a0050d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab90f3c6c4a7c80d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:53.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.106+0000 7f4985600640 1 -- 172.21.15.3:0/1757084326 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a00969a0 con 0x5555f6f6b900 2024-08-01T04:23:53.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.106+0000 7f4985600640 1 -- 172.21.15.3:0/1757084326 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f49a0096b40 con 0x5555f6f6b900 2024-08-01T04:23:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.107+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499402c900 con 0x5555f6f6b900 2024-08-01T04:23:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.107+0000 7f4985600640 1 -- 172.21.15.3:0/1757084326 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f49a0096d60 con 0x5555f6f6b900 2024-08-01T04:23:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.107+0000 7f49a90a8740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:53.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.107+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555f6f7a130 con 0x5555f6f6b900 2024-08-01T04:23:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.110+0000 7f4985600640 1 -- 172.21.15.3:0/1757084326 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f499402c900 con 0x5555f6f6b900 2024-08-01T04:23:53.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.110+0000 7f4985600640 1 -- 172.21.15.3:0/1757084326 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f49a0097660 con 0x5555f6f6b900 2024-08-01T04:23:53.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.110+0000 7f49a90a8740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:53.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.110+0000 7f49a90a8740 10 client.15918 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:23:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.110+0000 7f49a90a8740 20 client.15918 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.111+0000 7f49a90a8740 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6f7b970 0x5555f6f7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.111+0000 7f49a90a8740 4 client.15918 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:23:53.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.111+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f6ed2fe0 con 0x5555f6f7b970 2024-08-01T04:23:53.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.116+0000 7f499f400640 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6f7b970 0x5555f6f7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.120+0000 7f4985600640 10 client.15918 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:53.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.120+0000 7f499f400640 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6f7b970 0x5555f6f7dd80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499b91476250e745 server_cookie=c3db0cbf34dac599 in_seq=0 out_seq=0 2024-08-01T04:23:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.125+0000 7f4985600640 1 -- 172.21.15.3:0/1757084326 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5555f6ed2fe0 con 0x5555f6f7b970 2024-08-01T04:23:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.125+0000 7f4985600640 10 client.15918 handle_command_reply: tid=0 2024-08-01T04:23:53.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.125+0000 7f49a90a8740 1 client.15918 shutdown 2024-08-01T04:23:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 msgr2=0x5555f6f69280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f69280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f49a0065770 tx=0x7f49a0050d10 comp rx=0 tx=0).stop 2024-08-01T04:23:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 shutdown_connections 2024-08-01T04:23:53.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6f7b970 0x5555f6f7dd80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f69280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f68d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 --2- 172.21.15.3:0/1757084326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f4985600640 0 client.15918 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:53.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.126+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 >> 172.21.15.3:0/1757084326 conn(0x5555f6f5ccb0 msgr2=0x5555f6f5bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.127+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 shutdown_connections 2024-08-01T04:23:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.127+0000 7f49a90a8740 1 -- 172.21.15.3:0/1757084326 wait complete. 2024-08-01T04:23:53.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.127+0000 7f49a90a8740 20 client.15918 trim_cache size 0 max 16384 2024-08-01T04:23:53.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.183+0000 7f49a90a8740 1 Processor -- start 2024-08-01T04:23:53.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.183+0000 7f49a90a8740 1 -- start start 2024-08-01T04:23:53.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f83690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f49a90a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4998152bd0 con 0x5555f6f6b900 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f49a90a8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f49981504a0 con 0x5555f6f61040 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f49a90a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f49980a4c00 con 0x5555f6f60720 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.184+0000 7f499ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.185+0000 7f499f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.185+0000 7f499f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53240/0 (socket says 172.21.15.3:53240) 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.185+0000 7f499f400640 1 -- 172.21.15.3:0/4255959431 learned_addr learned my addr 172.21.15.3:0/4255959431 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:53.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.185+0000 7f499fe00640 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f83690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.186+0000 7f499ea00640 1 -- 172.21.15.3:0/4255959431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 msgr2=0x5555f6f690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.186+0000 7f499ea00640 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.186+0000 7f499ea00640 1 -- 172.21.15.3:0/4255959431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 msgr2=0x5555f6f83690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.186+0000 7f499ea00640 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f83690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.186+0000 7f499ea00640 1 -- 172.21.15.3:0/4255959431 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f6f677b0 con 0x5555f6f61040 2024-08-01T04:23:53.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.186+0000 7f499fe00640 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f83690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:23:53.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.187+0000 7f499ea00640 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6a660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f498c0ccf00 tx=0x7f498c0cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190e48478d1dd685 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.188+0000 7f4987e00640 1 -- 172.21.15.3:0/4255959431 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0b9070 con 0x5555f6f61040 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.188+0000 7f4987e00640 1 -- 172.21.15.3:0/4255959431 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c0a4940 con 0x5555f6f61040 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.188+0000 7f4987e00640 1 -- 172.21.15.3:0/4255959431 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c097a20 con 0x5555f6f61040 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 -- 172.21.15.3:0/4255959431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 msgr2=0x5555f6f6a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6a660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f498c0ccf00 tx=0x7f498c0cf040 comp rx=0 tx=0).stop 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 -- 172.21.15.3:0/4255959431 shutdown_connections 2024-08-01T04:23:53.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f6b900 0x5555f6f83690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f6a660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 --2- 172.21.15.3:0/4255959431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f60720 0x5555f6f690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.189+0000 7f49a90a8740 1 -- 172.21.15.3:0/4255959431 >> 172.21.15.3:0/4255959431 conn(0x5555f6f5ccb0 msgr2=0x5555f6f69f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.190+0000 7f49a90a8740 1 -- 172.21.15.3:0/4255959431 shutdown_connections 2024-08-01T04:23:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.190+0000 7f49a90a8740 1 -- 172.21.15.3:0/4255959431 wait complete. 2024-08-01T04:23:53.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.190+0000 7f49a90a8740 1 -- start start 2024-08-01T04:23:53.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.191+0000 7f49a90a8740 1 Processor -- start 2024-08-01T04:23:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 0x5555f6f6fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f6b900 0x5555f6fe5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49980b3b00 con 0x5555f6f60720 2024-08-01T04:23:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f499f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 0x5555f6f6fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f499814ff00 con 0x5555f6f61040 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f49980b4190 con 0x5555f6f6b900 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f499f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 0x5555f6f6fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60538/0 (socket says 172.21.15.3:60538) 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f499f400640 1 -- 172.21.15.3:0/3861361885 learned_addr learned my addr 172.21.15.3:0/3861361885 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f49a90a8740 10 client.0 _ll_register_callbacks cb 0x7f498c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.192+0000 7f499ea00640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499fe00640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f6b900 0x5555f6fe5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499f400640 1 -- 172.21.15.3:0/3861361885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f6b900 msgr2=0x5555f6fe5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499f400640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f6b900 0x5555f6fe5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499f400640 1 -- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 msgr2=0x5555f6f701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499f400640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499f400640 1 -- 172.21.15.3:0/3861361885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f6f60e70 con 0x5555f6f60720 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.193+0000 7f499fe00640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f6b900 0x5555f6fe5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.194+0000 7f499f400640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 0x5555f6f6fca0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5555f6f7e630 tx=0x7f4994005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d19114b7d1a6ae3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:23:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.194+0000 7f4985600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:23:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.195+0000 7f4985600640 1 -- 172.21.15.3:0/3861361885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994027040 con 0x5555f6f60720 2024-08-01T04:23:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.195+0000 7f4985600640 1 -- 172.21.15.3:0/3861361885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f4994005450 con 0x5555f6f60720 2024-08-01T04:23:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.195+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c0a4940 con 0x5555f6f60720 2024-08-01T04:23:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.195+0000 7f4985600640 1 -- 172.21.15.3:0/3861361885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f499403b040 con 0x5555f6f60720 2024-08-01T04:23:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.195+0000 7f49a90a8740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-08-01T04:23:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.195+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555f6fe6c50 con 0x5555f6f60720 2024-08-01T04:23:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f4985600640 1 -- 172.21.15.3:0/3861361885 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c0a4940 con 0x5555f6f60720 2024-08-01T04:23:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f4985600640 1 -- 172.21.15.3:0/3861361885 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f499402d980 con 0x5555f6f60720 2024-08-01T04:23:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f49a90a8740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:23:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f49a90a8740 10 client.15924 resolve_mds: validated gid 24548 aka daemon mds.b 2024-08-01T04:23:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f49a90a8740 20 client.15924 populate_metadata read hostname 'smithi003' 2024-08-01T04:23:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f49a90a8740 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6fe7050 0x5555f6fe9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:23:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f49a90a8740 4 client.15924 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-01T04:23:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.196+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5555f6ecd4f0 con 0x5555f6fe7050 2024-08-01T04:23:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.198+0000 7f499ea00640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6fe7050 0x5555f6fe9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:23:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.198+0000 7f499ea00640 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6fe7050 0x5555f6fe9440 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e23f8805ec5f24 server_cookie=255ad75dc877c324 in_seq=0 out_seq=0 2024-08-01T04:23:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.198+0000 7f4985600640 10 client.15924 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.201+0000 7f4985600640 1 -- 172.21.15.3:0/3861361885 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5555f6ecd4f0 con 0x5555f6fe7050 2024-08-01T04:23:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.201+0000 7f4985600640 10 client.15924 handle_command_reply: tid=0 2024-08-01T04:23:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.201+0000 7f49a90a8740 1 client.15924 shutdown 2024-08-01T04:23:53.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.205+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 msgr2=0x5555f6f6fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.205+0000 7f49a90a8740 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 0x5555f6f6fca0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5555f6f7e630 tx=0x7f4994005e10 comp rx=0 tx=0).stop 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.206+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 shutdown_connections 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.206+0000 7f49a90a8740 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x5555f6fe7050 0x5555f6fe9440 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.206+0000 7f49a90a8740 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5555f6f6b900 0x5555f6fe5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.206+0000 7f49a90a8740 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5555f6f61040 0x5555f6f701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.206+0000 7f49a90a8740 1 --2- 172.21.15.3:0/3861361885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555f6f60720 0x5555f6f6fca0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.206+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 >> 172.21.15.3:0/3861361885 conn(0x5555f6f5ccb0 msgr2=0x5555f6f6a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.207+0000 7f4985600640 0 client.15924 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:23:53.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.207+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 shutdown_connections 2024-08-01T04:23:53.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.207+0000 7f49a90a8740 1 -- 172.21.15.3:0/3861361885 wait complete. 2024-08-01T04:23:53.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.208+0000 7f49a90a8740 20 client.15924 trim_cache size 0 max 16384 2024-08-01T04:23:53.211 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:23:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 29, 2024-08-01T04:23:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24548, 2024-08-01T04:23:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 16, 2024-08-01T04:23:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": {} 2024-08-01T04:23:53.212 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:23:53.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.209+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49880a2ed0 msgr2=0x7f49880a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49880a2ed0 0x7f49880a5390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f499400abe0 tx=0x7f499400a2d0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 msgr2=0x7f49981438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:23:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 0x7f49981438c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e9f0 tx=0x7f498c00edf0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 shutdown_connections 2024-08-01T04:23:53.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f49880a2ed0 0x7f49880a5390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980b7540 0x7f4998147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49980a5140 0x7f49981438c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 --2- 172.21.15.3:0/2659511881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49980a4820 0x7f4998143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:23:53.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.210+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 >> 172.21.15.3:0/2659511881 conn(0x7f499801a0e0 msgr2=0x7f49980b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:23:53.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.211+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 shutdown_connections 2024-08-01T04:23:53.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.211+0000 7f49a5000640 1 -- 172.21.15.3:0/2659511881 wait complete. 2024-08-01T04:23:53.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:53 smithi003 ceph-mon[32262]: pgmap v233: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 34 KiB/s rd, 48 MiB/s wr, 7.43k op/s 2024-08-01T04:23:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:53 smithi160 ceph-mon[37224]: pgmap v233: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 34 KiB/s rd, 48 MiB/s wr, 7.43k op/s 2024-08-01T04:23:54.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:53 smithi163 ceph-mon[37383]: pgmap v233: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 34 KiB/s rd, 48 MiB/s wr, 7.43k op/s 2024-08-01T04:23:54.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:54 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:23:54.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:54 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:54.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:54 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:23:55.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:54 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:23:55.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:54 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:55.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:54 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:23:55.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:54 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:23:55.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:54 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:55.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:54 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:23:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:55 smithi160 ceph-mon[37224]: pgmap v234: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 6.33k op/s 2024-08-01T04:23:56.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:55 smithi163 ceph-mon[37383]: pgmap v234: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 6.33k op/s 2024-08-01T04:23:56.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:55 smithi003 ceph-mon[32262]: pgmap v234: 97 pgs: 97 active+clean; 27 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 6.33k op/s 2024-08-01T04:23:57.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:57 smithi160 ceph-mon[37224]: pgmap v235: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 MiB/s wr, 7.54k op/s 2024-08-01T04:23:58.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:57 smithi163 ceph-mon[37383]: pgmap v235: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 MiB/s wr, 7.54k op/s 2024-08-01T04:23:58.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:57 smithi003 ceph-mon[32262]: pgmap v235: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 49 KiB/s rd, 45 MiB/s wr, 7.54k op/s 2024-08-01T04:23:58.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:58 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:23:58.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:58 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:58.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:58 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:23:58.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:58 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:58.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:58 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:23:58.800 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:58 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:58.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:58 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:23:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:58 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:58 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:23:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:58 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:58 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:23:58.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:58 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:59.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:58 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:23:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:58 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:58 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:23:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:58 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:58 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:23:59.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:58 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:59 smithi160 ceph-mon[37224]: pgmap v236: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 28 KiB/s rd, 43 MiB/s wr, 7.60k op/s 2024-08-01T04:23:59.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:59 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:23:59.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:59 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:23:59.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:23:59 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:00.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:59 smithi003 ceph-mon[32262]: pgmap v236: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 28 KiB/s rd, 43 MiB/s wr, 7.60k op/s 2024-08-01T04:24:00.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:59 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:24:00.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:59 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:00.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:23:59 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:00.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:59 smithi163 ceph-mon[37383]: pgmap v236: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 28 KiB/s rd, 43 MiB/s wr, 7.60k op/s 2024-08-01T04:24:00.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:59 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:24:00.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:59 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:00.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:23:59 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:00.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.281+0000 7faaf2e00640 1 -- 172.21.15.3:0/2052901719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40aab50 msgr2=0x7faae40a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.281+0000 7faaf2e00640 1 --2- 172.21.15.3:0/2052901719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40aab50 0x7faae40a47f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faadc009900 tx=0x7faadc02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.284+0000 7faaf2e00640 1 -- 172.21.15.3:0/2052901719 shutdown_connections 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.284+0000 7faaf2e00640 1 --2- 172.21.15.3:0/2052901719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40a4d30 0x7faae40a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.284+0000 7faaf2e00640 1 --2- 172.21.15.3:0/2052901719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40aab50 0x7faae40a47f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.284+0000 7faaf2e00640 1 --2- 172.21.15.3:0/2052901719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aa230 0x7faae40aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.284+0000 7faaf2e00640 1 -- 172.21.15.3:0/2052901719 >> 172.21.15.3:0/2052901719 conn(0x7faae401a0a0 msgr2=0x7faae401a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.285+0000 7faaf2e00640 1 -- 172.21.15.3:0/2052901719 shutdown_connections 2024-08-01T04:24:00.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.287+0000 7faaf2e00640 1 -- 172.21.15.3:0/2052901719 wait complete. 2024-08-01T04:24:00.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.288+0000 7faaf2e00640 1 Processor -- start 2024-08-01T04:24:00.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.290+0000 7faaf2e00640 1 -- start start 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.292+0000 7faaf2e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 0x7faae4143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.293+0000 7faaf2e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 0x7faae41437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.294+0000 7faaf2e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 0x7faae4147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.294+0000 7faaebe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 0x7faae4143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.295+0000 7faaeb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 0x7faae41437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.295+0000 7faaf0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 0x7faae4147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.296+0000 7faaeb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 0x7faae41437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52036/0 (socket says 172.21.15.3:52036) 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaebe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 0x7faae4143270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53272/0 (socket says 172.21.15.3:53272) 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf2e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faae40bcff0 con 0x7faae40aab50 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf2e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faae40bce70 con 0x7faae40aa230 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf2e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faae40bd170 con 0x7faae40a4d30 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 0x7faae4147b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60556/0 (socket says 172.21.15.3:60556) 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 -- 172.21.15.3:0/1187379368 learned_addr learned my addr 172.21.15.3:0/1187379368 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:00.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 -- 172.21.15.3:0/1187379368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 msgr2=0x7faae4143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 0x7faae4143270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 -- 172.21.15.3:0/1187379368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 msgr2=0x7faae41437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 0x7faae41437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.297+0000 7faaf0e00640 1 -- 172.21.15.3:0/1187379368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae4148220 con 0x7faae40aab50 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.298+0000 7faaeb400640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 0x7faae41437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.298+0000 7faaebe00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 0x7faae4143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-01T04:24:00.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.299+0000 7faaf0e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 0x7faae4147b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faaec0515e0 tx=0x7faaec073b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.302+0000 7faae8c00640 1 -- 172.21.15.3:0/1187379368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec079070 con 0x7faae40aab50 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.302+0000 7faae8c00640 1 -- 172.21.15.3:0/1187379368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec075430 con 0x7faae40aab50 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.302+0000 7faae8c00640 1 -- 172.21.15.3:0/1187379368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec074620 con 0x7faae40aab50 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.302+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae4148510 con 0x7faae40aab50 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.302+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae414fdf0 con 0x7faae40aab50 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.321+0000 7faaf5844740 1 Processor -- start 2024-08-01T04:24:00.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.325+0000 7faae8c00640 1 -- 172.21.15.3:0/1187379368 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7faaec07c070 con 0x7faae40aab50 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.325+0000 7faaf5844740 1 -- start start 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.326+0000 7faaf5844740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c1f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.326+0000 7faaf5844740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c2022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.326+0000 7faaf5844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c204c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.326+0000 7faaf5844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d41c1127e0 con 0x55d41c202830 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.326+0000 7faaf5844740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d41c15dac0 con 0x55d41c1f7f70 2024-08-01T04:24:00.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.326+0000 7faaf5844740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d41c0f58c0 con 0x55d41c1f7650 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.327+0000 7faaeb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c1f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.327+0000 7faaeb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c1f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53280/0 (socket says 172.21.15.3:53280) 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.327+0000 7faaeb400640 1 -- 172.21.15.3:0/3666528529 learned_addr learned my addr 172.21.15.3:0/3666528529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.327+0000 7faae8c00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7faad40a2e50 0x7faad40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.328+0000 7faae8c00640 1 -- 172.21.15.3:0/1187379368 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7faaec133040 con 0x7faae40aab50 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.328+0000 7faaeb400640 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c204c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.329+0000 7faaf0e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7faad40a2e50 0x7faad40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.330+0000 7faaeb400640 1 -- 172.21.15.3:0/3666528529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 msgr2=0x55d41c2022f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:24:00.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.330+0000 7faaeb400640 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c2022f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.330+0000 7faaeb400640 1 -- 172.21.15.3:0/3666528529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 msgr2=0x55d41c204c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:00.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.330+0000 7faaeb400640 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c204c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.333+0000 7faaeb400640 1 -- 172.21.15.3:0/3666528529 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41c0fc760 con 0x55d41c1f7650 2024-08-01T04:24:00.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.333+0000 7faaeb400640 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c1f7a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faadc00b230 tx=0x7faadc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6ea4ede461970a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.334+0000 7faad9600640 1 -- 172.21.15.3:0/3666528529 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc042020 con 0x55d41c1f7650 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.334+0000 7faad9600640 1 -- 172.21.15.3:0/3666528529 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc032dd0 con 0x55d41c1f7650 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.334+0000 7faad9600640 1 -- 172.21.15.3:0/3666528529 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035040 con 0x55d41c1f7650 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.336+0000 7faaf5844740 1 -- 172.21.15.3:0/3666528529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 msgr2=0x55d41c1f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.337+0000 7faaf5844740 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c1f7a30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faadc00b230 tx=0x7faadc0056d0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.337+0000 7faaf5844740 1 -- 172.21.15.3:0/3666528529 shutdown_connections 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.338+0000 7faaf5844740 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c204c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.338+0000 7faaf5844740 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c2022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.338+0000 7faaf5844740 1 --2- 172.21.15.3:0/3666528529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c1f7a30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.339+0000 7faaf5844740 1 -- 172.21.15.3:0/3666528529 >> 172.21.15.3:0/3666528529 conn(0x55d41c1f3be0 msgr2=0x55d41c1f3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.339+0000 7faaf0e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7faad40a2e50 0x7faad40a5310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faaec1307d0 tx=0x7faaec0689e0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.339+0000 7faaf5844740 1 -- 172.21.15.3:0/3666528529 shutdown_connections 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.339+0000 7faaf5844740 1 -- 172.21.15.3:0/3666528529 wait complete. 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.341+0000 7faaf5844740 1 -- start start 2024-08-01T04:24:00.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.343+0000 7faaf5844740 1 Processor -- start 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c1ff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c1ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d41c118ae0 con 0x55d41c1f7650 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55d41c159a80 con 0x55d41c1f7f70 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55d41c216500 con 0x55d41c202830 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaf5844740 10 client.0 _ll_register_callbacks cb 0x55d41c11a190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:00.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c1ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c1ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52046/0 (socket says 172.21.15.3:52046) 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.349+0000 7faaebe00640 1 -- 172.21.15.3:0/960963836 learned_addr learned my addr 172.21.15.3:0/960963836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.350+0000 7faaebe00640 1 -- 172.21.15.3:0/960963836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 msgr2=0x55d41c2001a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.350+0000 7faaebe00640 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2001a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.350+0000 7faaebe00640 1 -- 172.21.15.3:0/960963836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 msgr2=0x55d41c1ff720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.350+0000 7faaebe00640 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c1ff720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.350+0000 7faaebe00640 1 -- 172.21.15.3:0/960963836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41becf6a0 con 0x55d41c1f7f70 2024-08-01T04:24:00.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.351+0000 7faaebe00640 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c1ffc60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faae000d490 tx=0x7faae000d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d75e699d91598d6e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.352+0000 7faace000640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.352+0000 7faace000640 1 -- 172.21.15.3:0/960963836 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae000f8f0 con 0x55d41c1f7f70 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.353+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faadc032dd0 con 0x55d41c1f7f70 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.353+0000 7faace000640 1 -- 172.21.15.3:0/960963836 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7faae000f6f0 con 0x55d41c1f7f70 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.354+0000 7faace000640 1 -- 172.21.15.3:0/960963836 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faae000f8f0 con 0x55d41c1f7f70 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.355+0000 7faaf5844740 10 client.26111 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.355+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d41c222160 con 0x55d41c1f7f70 2024-08-01T04:24:00.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.357+0000 7faace000640 1 -- 172.21.15.3:0/960963836 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc032dd0 con 0x55d41c1f7f70 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faace000640 1 -- 172.21.15.3:0/960963836 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7faae000e410 con 0x55d41c1f7f70 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faaf5844740 10 client.26111 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faaf5844740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faaf5844740 20 client.26111 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faaf5844740 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c223500 0x55d41c225980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faaf5844740 4 client.26111 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:00.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.361+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d41c16b030 con 0x55d41c223500 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.364+0000 7faaeb400640 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c223500 0x55d41c225980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.364+0000 7faaeb400640 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c223500 0x55d41c225980 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62dfe56d2752446 server_cookie=8f1330372d3a7333 in_seq=0 out_seq=0 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.365+0000 7faace000640 10 client.26111 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.368+0000 7faace000640 1 -- 172.21.15.3:0/960963836 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d41c16b030 con 0x55d41c223500 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.368+0000 7faace000640 10 client.26111 handle_command_reply: tid=0 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.368+0000 7faaf5844740 1 client.26111 shutdown 2024-08-01T04:24:00.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.368+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 msgr2=0x55d41c1ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.368+0000 7faaf5844740 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c1ffc60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faae000d490 tx=0x7faae000d950 comp rx=0 tx=0).stop 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 shutdown_connections 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faaf5844740 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c223500 0x55d41c225980 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faaf5844740 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faaf5844740 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1f7f70 0x55d41c1ffc60 secure :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faae000d490 tx=0x7faae000d950 comp rx=0 tx=0).stop 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faaf5844740 1 --2- 172.21.15.3:0/960963836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c1ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 >> 172.21.15.3:0/960963836 conn(0x55d41c1f3be0 msgr2=0x55d41c1f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.369+0000 7faace000640 0 client.26111 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.371+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 shutdown_connections 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.371+0000 7faaf5844740 1 -- 172.21.15.3:0/960963836 wait complete. 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.372+0000 7faaf5844740 20 client.26111 trim_cache size 0 max 16384 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.429+0000 7faaf5844740 1 Processor -- start 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.430+0000 7faaf5844740 1 -- start start 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.431+0000 7faaf5844740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c2015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.431+0000 7faaf5844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c1ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.431+0000 7faaf5844740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c1f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.431+0000 7faaf5844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faae4152a40 con 0x55d41c202830 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.431+0000 7faaf5844740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faae4150310 con 0x55d41c1fe810 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.431+0000 7faaf5844740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faae41445d0 con 0x55d41c1f7650 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.433+0000 7faaeb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c2015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.433+0000 7faaeb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c2015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53286/0 (socket says 172.21.15.3:53286) 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.433+0000 7faaeb400640 1 -- 172.21.15.3:0/1303300379 learned_addr learned my addr 172.21.15.3:0/1303300379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:00.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.433+0000 7faaeb400640 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c1f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.434+0000 7faaebe00640 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c1ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.436+0000 7faaeb400640 1 -- 172.21.15.3:0/1303300379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 msgr2=0x55d41c1f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.436+0000 7faaeb400640 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c1f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.436+0000 7faaeb400640 1 -- 172.21.15.3:0/1303300379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 msgr2=0x55d41c1ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.436+0000 7faaeb400640 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c1ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.438+0000 7faaeb400640 1 -- 172.21.15.3:0/1303300379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41c226000 con 0x55d41c1f7650 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.439+0000 7faaeb400640 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c2015f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faadc033000 tx=0x7faadc040040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43649d5ff7a1fd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.440+0000 7faad8c00640 1 -- 172.21.15.3:0/1303300379 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035050 con 0x55d41c1f7650 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.440+0000 7faad8c00640 1 -- 172.21.15.3:0/1303300379 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc032460 con 0x55d41c1f7650 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.440+0000 7faad8c00640 1 -- 172.21.15.3:0/1303300379 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035050 con 0x55d41c1f7650 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.443+0000 7faaf5844740 1 -- 172.21.15.3:0/1303300379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 msgr2=0x55d41c2015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.443+0000 7faaf5844740 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c2015f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faadc033000 tx=0x7faadc040040 comp rx=0 tx=0).stop 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.444+0000 7faaf5844740 1 -- 172.21.15.3:0/1303300379 shutdown_connections 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.444+0000 7faaf5844740 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.444+0000 7faaf5844740 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c202830 0x55d41c1ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.444+0000 7faaf5844740 1 --2- 172.21.15.3:0/1303300379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c1f7650 0x55d41c2015f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.445+0000 7faaf5844740 1 -- 172.21.15.3:0/1303300379 >> 172.21.15.3:0/1303300379 conn(0x55d41c1f3be0 msgr2=0x55d41c1f2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.445+0000 7faaf5844740 1 -- 172.21.15.3:0/1303300379 shutdown_connections 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.445+0000 7faaf5844740 1 -- 172.21.15.3:0/1303300379 wait complete. 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.446+0000 7faaf5844740 1 -- start start 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.447+0000 7faaf5844740 1 Processor -- start 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c21b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c21bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faae40b49f0 con 0x55d41c1f7650 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faae40b5df0 con 0x55d41c1fe810 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7faae40b5050 con 0x55d41c202830 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.452+0000 7faaf5844740 10 client.0 _ll_register_callbacks cb 0x7faae0005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.453+0000 7faaebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c21bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.454+0000 7faaebe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c21bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52064/0 (socket says 172.21.15.3:52064) 2024-08-01T04:24:00.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.454+0000 7faaebe00640 1 -- 172.21.15.3:0/4250561210 learned_addr learned my addr 172.21.15.3:0/4250561210 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.455+0000 7faaeb400640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.455+0000 7faaeb400640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c21b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.456+0000 7faaeb400640 1 -- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 msgr2=0x55d41c21bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.456+0000 7faaeb400640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c21bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.456+0000 7faaeb400640 1 -- 172.21.15.3:0/4250561210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 msgr2=0x55d41c21b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.456+0000 7faaeb400640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c21b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.456+0000 7faaeb400640 1 -- 172.21.15.3:0/4250561210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d41c218fd0 con 0x55d41c202830 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.459+0000 7faaeb400640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2187a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faadc001eb0 tx=0x7faadc005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773d7275e9573aaa server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.459+0000 7faace000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.460+0000 7faace000640 1 -- 172.21.15.3:0/4250561210 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc035040 con 0x55d41c202830 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.460+0000 7faace000640 1 -- 172.21.15.3:0/4250561210 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7faadc032260 con 0x55d41c202830 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.460+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faadc032460 con 0x55d41c202830 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.461+0000 7faace000640 1 -- 172.21.15.3:0/4250561210 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faadc042040 con 0x55d41c202830 2024-08-01T04:24:00.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.462+0000 7faaf5844740 10 client.26092 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.462+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d41c28edd0 con 0x55d41c202830 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.462+0000 7faace000640 1 -- 172.21.15.3:0/4250561210 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc032460 con 0x55d41c202830 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faace000640 1 -- 172.21.15.3:0/4250561210 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7faadc03a040 con 0x55d41c202830 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faaf5844740 10 client.26092 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faaf5844740 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faaf5844740 20 client.26092 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faaf5844740 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c28f050 0x55d41c291440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faaf5844740 4 client.26092 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.464+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d41c165540 con 0x55d41c28f050 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.465+0000 7faaebe00640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c28f050 0x55d41c291440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.466+0000 7faaebe00640 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c28f050 0x55d41c291440 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff40e455fdc82ad server_cookie=419862647066e6c9 in_seq=0 out_seq=0 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.466+0000 7faace000640 10 client.26092 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.470+0000 7faace000640 1 -- 172.21.15.3:0/4250561210 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d41c165540 con 0x55d41c28f050 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.470+0000 7faace000640 10 client.26092 handle_command_reply: tid=0 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.470+0000 7faaf5844740 1 client.26092 shutdown 2024-08-01T04:24:00.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.471+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 msgr2=0x55d41c2187a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.471+0000 7faaf5844740 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2187a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faadc001eb0 tx=0x7faadc005c90 comp rx=0 tx=0).stop 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.471+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 shutdown_connections 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.472+0000 7faaf5844740 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55d41c28f050 0x55d41c291440 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.472+0000 7faaf5844740 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d41c202830 0x55d41c2187a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.472+0000 7faaf5844740 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d41c1fe810 0x55d41c21bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.472+0000 7faaf5844740 1 --2- 172.21.15.3:0/4250561210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d41c1f7650 0x55d41c21b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.472+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 >> 172.21.15.3:0/4250561210 conn(0x55d41c1f3be0 msgr2=0x55d41c1f2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.472+0000 7faace000640 0 client.26092 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.473+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 shutdown_connections 2024-08-01T04:24:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.473+0000 7faaf5844740 1 -- 172.21.15.3:0/4250561210 wait complete. 2024-08-01T04:24:00.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.473+0000 7faaf5844740 20 client.26092 trim_cache size 0 max 16384 2024-08-01T04:24:00.484 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:24:00.485 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:24:00.485 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "51e75c29-472d-47d4-80b3-a9600402b771", 2024-08-01T04:24:00.485 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:24:00.485 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:24:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.477+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7faad40a2e50 msgr2=0x7faad40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.477+0000 7faaf2e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7faad40a2e50 0x7faad40a5310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faaec1307d0 tx=0x7faaec0689e0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.477+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 msgr2=0x7faae4147b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.477+0000 7faaf2e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 0x7faae4147b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faaec0515e0 tx=0x7faaec073b70 comp rx=0 tx=0).stop 2024-08-01T04:24:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.478+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 shutdown_connections 2024-08-01T04:24:00.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.478+0000 7faaf2e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faae40aab50 0x7faae4147b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.478+0000 7faaf2e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae40aa230 0x7faae41437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.478+0000 7faaf2e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7faad40a2e50 0x7faad40a5310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.478+0000 7faaf2e00640 1 --2- 172.21.15.3:0/1187379368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae40a4d30 0x7faae4143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.478+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 >> 172.21.15.3:0/1187379368 conn(0x7faae401a0a0 msgr2=0x7faae40a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.480+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 shutdown_connections 2024-08-01T04:24:00.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:23:53.480+0000 7faaf2e00640 1 -- 172.21.15.3:0/1187379368 wait complete. 2024-08-01T04:24:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.562+0000 7fa28ae00640 1 -- 172.21.15.3:0/2777542019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284100f10 msgr2=0x7fa2841056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.562+0000 7fa28ae00640 1 --2- 172.21.15.3:0/2777542019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284100f10 0x7fa2841056b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009900 tx=0x7fa26c02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 -- 172.21.15.3:0/2777542019 shutdown_connections 2024-08-01T04:24:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 --2- 172.21.15.3:0/2777542019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284105bf0 0x7fa284106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 --2- 172.21.15.3:0/2777542019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284100f10 0x7fa2841056b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 --2- 172.21.15.3:0/2777542019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 0x7fa2841009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 -- 172.21.15.3:0/2777542019 >> 172.21.15.3:0/2777542019 conn(0x7fa2840fc500 msgr2=0x7fa2840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 -- 172.21.15.3:0/2777542019 shutdown_connections 2024-08-01T04:24:00.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.563+0000 7fa28ae00640 1 -- 172.21.15.3:0/2777542019 wait complete. 2024-08-01T04:24:00.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.564+0000 7fa28ae00640 1 Processor -- start 2024-08-01T04:24:00.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.564+0000 7fa28ae00640 1 -- start start 2024-08-01T04:24:00.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa28ae00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 0x7fa2841a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa28ae00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 0x7fa2841a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa28ae00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284105bf0 0x7fa28419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa28ae00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841183b0 con 0x7fa284105bf0 2024-08-01T04:24:00.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa28ae00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa284118230 con 0x7fa284100f10 2024-08-01T04:24:00.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa28ae00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa284118530 con 0x7fa2841005f0 2024-08-01T04:24:00.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa283400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 0x7fa2841a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa283e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 0x7fa2841a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa283400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 0x7fa2841a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:52084/0 (socket says 172.21.15.3:52084) 2024-08-01T04:24:00.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa283400640 1 -- 172.21.15.3:0/186944781 learned_addr learned my addr 172.21.15.3:0/186944781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa283e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 0x7fa2841a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:53306/0 (socket says 172.21.15.3:53306) 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.565+0000 7fa288e00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284105bf0 0x7fa28419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.566+0000 7fa283400640 1 -- 172.21.15.3:0/186944781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 msgr2=0x7fa2841a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.566+0000 7fa283400640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 0x7fa2841a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.566+0000 7fa283400640 1 -- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284105bf0 msgr2=0x7fa28419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.566+0000 7fa283400640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284105bf0 0x7fa28419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.566+0000 7fa283400640 1 -- 172.21.15.3:0/186944781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28419efd0 con 0x7fa284100f10 2024-08-01T04:24:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.567+0000 7fa283400640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 0x7fa2841a4bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02a9e0 tx=0x7fa26c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.567+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c03f070 con 0x7fa284100f10 2024-08-01T04:24:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.567+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c031c30 con 0x7fa284100f10 2024-08-01T04:24:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.567+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035030 con 0x7fa284100f10 2024-08-01T04:24:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.567+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28419f260 con 0x7fa284100f10 2024-08-01T04:24:00.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.567+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2841ab440 con 0x7fa284100f10 2024-08-01T04:24:00.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.569+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28406a580 con 0x7fa284100f10 2024-08-01T04:24:00.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.571+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa26c042070 con 0x7fa284100f10 2024-08-01T04:24:00.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.573+0000 7fa280c00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa2680a2d70 0x7fa2680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:00.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.573+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fa26c0ef710 con 0x7fa284100f10 2024-08-01T04:24:00.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.573+0000 7fa283e00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa2680a2d70 0x7fa2680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:00.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.574+0000 7fa283e00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa2680a2d70 0x7fa2680a5230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa274005fd0 tx=0x7fa274009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:00.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.576+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa26c0bc260 con 0x7fa284100f10 2024-08-01T04:24:00.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.831+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2841ab610 con 0x7fa284100f10 2024-08-01T04:24:00.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.832+0000 7fa280c00640 1 -- 172.21.15.3:0/186944781 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7fa26c031de0 con 0x7fa284100f10 2024-08-01T04:24:00.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:24:00.835 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:24:00.836 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:24:00.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa2680a2d70 msgr2=0x7fa2680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa2680a2d70 0x7fa2680a5230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa274005fd0 tx=0x7fa274009500 comp rx=0 tx=0).stop 2024-08-01T04:24:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 msgr2=0x7fa2841a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 0x7fa2841a4bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02a9e0 tx=0x7fa26c003c10 comp rx=0 tx=0).stop 2024-08-01T04:24:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 shutdown_connections 2024-08-01T04:24:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa284105bf0 0x7fa28419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa284100f10 0x7fa2841a4bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa2680a2d70 0x7fa2680a5230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 --2- 172.21.15.3:0/186944781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2841005f0 0x7fa2841a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 >> 172.21.15.3:0/186944781 conn(0x7fa2840fc500 msgr2=0x7fa28410f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.837+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 shutdown_connections 2024-08-01T04:24:00.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:00.838+0000 7fa28ae00640 1 -- 172.21.15.3:0/186944781 wait complete. 2024-08-01T04:24:01.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:01 smithi003 ceph-mon[32262]: pgmap v237: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 42 KiB/s rd, 44 MiB/s wr, 7.95k op/s 2024-08-01T04:24:01.790 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:01 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/186944781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:24:01.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:01 smithi160 ceph-mon[37224]: pgmap v237: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 42 KiB/s rd, 44 MiB/s wr, 7.95k op/s 2024-08-01T04:24:01.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:01 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/186944781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:24:02.046 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:24:02.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:01 smithi163 ceph-mon[37383]: pgmap v237: 97 pgs: 97 active+clean; 27 GiB data, 54 GiB used, 1018 GiB / 1.0 TiB avail; 42 KiB/s rd, 44 MiB/s wr, 7.95k op/s 2024-08-01T04:24:02.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:01 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/186944781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:24:02.678 INFO:tasks.quiescer:done joining Quiescers 2024-08-01T04:24:02.678 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-01T04:24:02.690 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-01T04:24:02.690 INFO:teuthology.task.exec:Executing custom commands... 2024-08-01T04:24:02.690 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:24:02.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-01T04:24:02.725 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:02.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-01T04:24:03.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 -- 172.21.15.3:0/3577756533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc578102090 msgr2=0x7fc57810e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 --2- 172.21.15.3:0/3577756533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc578102090 0x7fc57810e5c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc570009cd0 tx=0x7fc57002a8f0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 -- 172.21.15.3:0/3577756533 shutdown_connections 2024-08-01T04:24:03.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 --2- 172.21.15.3:0/3577756533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc578102090 0x7fc57810e5c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 --2- 172.21.15.3:0/3577756533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5781016f0 0x7fc578101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 --2- 172.21.15.3:0/3577756533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 0x7fc578107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 -- 172.21.15.3:0/3577756533 >> 172.21.15.3:0/3577756533 conn(0x7fc5780fd470 msgr2=0x7fc5780ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 -- 172.21.15.3:0/3577756533 shutdown_connections 2024-08-01T04:24:03.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.402+0000 7fc57ec00640 1 -- 172.21.15.3:0/3577756533 wait complete. 2024-08-01T04:24:03.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.404+0000 7fc57ec00640 1 Processor -- start 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.404+0000 7fc57ec00640 1 -- start start 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.404+0000 7fc57ec00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 0x7fc578197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.404+0000 7fc57ec00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc578102090 0x7fc578198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57ec00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 0x7fc57819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57ec00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc578111b40 con 0x7fc5781016f0 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57ec00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781119c0 con 0x7fc578102090 2024-08-01T04:24:03.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57ec00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc578111cc0 con 0x7fc5781076f0 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 0x7fc578197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc578102090 0x7fc578198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 0x7fc578197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37484/0 (socket says 172.21.15.3:37484) 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57d800640 1 -- 172.21.15.3:0/1162199873 learned_addr learned my addr 172.21.15.3:0/1162199873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57e200640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 0x7fc57819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.405+0000 7fc57d800640 1 -- 172.21.15.3:0/1162199873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 msgr2=0x7fc57819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57d800640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 0x7fc57819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57d800640 1 -- 172.21.15.3:0/1162199873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc578102090 msgr2=0x7fc578198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57d800640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc578102090 0x7fc578198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57d800640 1 -- 172.21.15.3:0/1162199873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57819cf70 con 0x7fc5781016f0 2024-08-01T04:24:03.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57ce00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc578102090 0x7fc578198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:03.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57e200640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 0x7fc57819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:03.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57d800640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 0x7fc578197fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00c600 tx=0x7fc56c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:03.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc576000640 1 -- 172.21.15.3:0/1162199873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c005020 con 0x7fc5781016f0 2024-08-01T04:24:03.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc576000640 1 -- 172.21.15.3:0/1162199873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56c0056a0 con 0x7fc5781016f0 2024-08-01T04:24:03.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc576000640 1 -- 172.21.15.3:0/1162199873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c01e850 con 0x7fc5781016f0 2024-08-01T04:24:03.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.406+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57819d260 con 0x7fc5781016f0 2024-08-01T04:24:03.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.407+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc578074150 con 0x7fc5781016f0 2024-08-01T04:24:03.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.410+0000 7fc576000640 1 -- 172.21.15.3:0/1162199873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc56c0046c0 con 0x7fc5781016f0 2024-08-01T04:24:03.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.411+0000 7fc576000640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc5600a2d70 0x7fc5600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.411+0000 7fc576000640 1 -- 172.21.15.3:0/1162199873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fc56c0d77a0 con 0x7fc5781016f0 2024-08-01T04:24:03.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.412+0000 7fc57ce00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc5600a2d70 0x7fc5600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.413+0000 7fc57ce00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc5600a2d70 0x7fc5600a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc5780ffa80 tx=0x7fc564012040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:03.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 Processor -- start 2024-08-01T04:24:03.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 -- start start 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c279e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 0x55e9c27a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e9c2703bf0 con 0x55e9c279dd80 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e9c269bbd0 con 0x55e9c27a8f60 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.421+0000 7fc582dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e9c26ffbb0 con 0x55e9c279e6a0 2024-08-01T04:24:03.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 0x55e9c27a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c279e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 0x55e9c27a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50886/0 (socket says 172.21.15.3:50886) 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 -- 172.21.15.3:0/2554684869 learned_addr learned my addr 172.21.15.3:0/2554684869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57e200640 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 -- 172.21.15.3:0/2554684869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 msgr2=0x55e9c27ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 -- 172.21.15.3:0/2554684869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 msgr2=0x55e9c279e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c279e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 -- 172.21.15.3:0/2554684869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c26a28a0 con 0x55e9c279e6a0 2024-08-01T04:24:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.422+0000 7fc57ce00640 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 0x55e9c27a8a20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc564004570 tx=0x7fc564014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d612033318e092d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:03.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.423+0000 7fc56aa00640 1 -- 172.21.15.3:0/2554684869 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564026470 con 0x55e9c279e6a0 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.423+0000 7fc56aa00640 1 -- 172.21.15.3:0/2554684869 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564021070 con 0x55e9c279e6a0 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.423+0000 7fc56aa00640 1 -- 172.21.15.3:0/2554684869 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56401d830 con 0x55e9c279e6a0 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.423+0000 7fc582dc2740 1 -- 172.21.15.3:0/2554684869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 msgr2=0x55e9c27a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.423+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 0x55e9c27a8a20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc564004570 tx=0x7fc564014000 comp rx=0 tx=0).stop 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 -- 172.21.15.3:0/2554684869 shutdown_connections 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279e6a0 0x55e9c27a8a20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2554684869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c279e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 -- 172.21.15.3:0/2554684869 >> 172.21.15.3:0/2554684869 conn(0x55e9c279a310 msgr2=0x55e9c279a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 -- 172.21.15.3:0/2554684869 shutdown_connections 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 -- 172.21.15.3:0/2554684869 wait complete. 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.424+0000 7fc582dc2740 1 -- start start 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 Processor -- start 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c27a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27a6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c27a8f60 0x55e9c27a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e9c27abb30 con 0x55e9c279dd80 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e9c26bed40 con 0x55e9c279e6a0 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e9c27abcb0 con 0x55e9c27a8f60 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.425+0000 7fc582dc2740 10 client.0 _ll_register_callbacks cb 0x55e9c2687b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c27a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c27a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37508/0 (socket says 172.21.15.3:37508) 2024-08-01T04:24:03.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 -- 172.21.15.3:0/4276645033 learned_addr learned my addr 172.21.15.3:0/4276645033 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:03.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57e200640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c27a8f60 0x55e9c27a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 -- 172.21.15.3:0/4276645033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c27a8f60 msgr2=0x55e9c27a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c27a8f60 0x55e9c27a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 -- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 msgr2=0x55e9c27a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 -- 172.21.15.3:0/4276645033 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c24753c0 con 0x55e9c279dd80 2024-08-01T04:24:03.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57e200640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c27a8f60 0x55e9c27a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57ce00640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc57d800640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c27a5e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc56c004e20 tx=0x7fc56c0a3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e92f1f10864141 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.426+0000 7fc55f400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc55f400640 1 -- 172.21.15.3:0/4276645033 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0ba070 con 0x55e9c279dd80 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc55f400640 1 -- 172.21.15.3:0/4276645033 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fc56c0a0310 con 0x55e9c279dd80 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc564021070 con 0x55e9c279dd80 2024-08-01T04:24:03.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc55f400640 1 -- 172.21.15.3:0/4276645033 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc56c0c2370 con 0x55e9c279dd80 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc582dc2740 10 client.15948 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9c27b7840 con 0x55e9c279dd80 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.427+0000 7fc55f400640 1 -- 172.21.15.3:0/4276645033 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564021070 con 0x55e9c279dd80 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc55f400640 1 -- 172.21.15.3:0/4276645033 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc56c09f9f0 con 0x55e9c279dd80 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc582dc2740 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc582dc2740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc582dc2740 20 client.15948 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc582dc2740 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c27b8b90 0x55e9c27bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc582dc2740 4 client.15948 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9c2711160 con 0x55e9c27b8b90 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.428+0000 7fc57ce00640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c27b8b90 0x55e9c27bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.429+0000 7fc57ce00640 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c27b8b90 0x55e9c27bafa0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f4426b049aa638 server_cookie=c4c25aad54981752 in_seq=0 out_seq=0 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.429+0000 7fc55f400640 10 client.15948 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc55f400640 1 -- 172.21.15.3:0/4276645033 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e9c2711160 con 0x55e9c27b8b90 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc55f400640 10 client.15948 handle_command_reply: tid=0 2024-08-01T04:24:03.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 client.15948 shutdown 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 msgr2=0x55e9c27a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c27a5e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc56c004e20 tx=0x7fc56c0a3de0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 shutdown_connections 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c27b8b90 0x55e9c27bafa0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c27a8f60 0x55e9c27a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc55f400640 0 client.15948 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 --2- 172.21.15.3:0/4276645033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279dd80 0x55e9c27a5e50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.430+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 >> 172.21.15.3:0/4276645033 conn(0x55e9c279a310 msgr2=0x55e9c27a0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.431+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 shutdown_connections 2024-08-01T04:24:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.431+0000 7fc582dc2740 1 -- 172.21.15.3:0/4276645033 wait complete. 2024-08-01T04:24:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.431+0000 7fc582dc2740 20 client.15948 trim_cache size 0 max 16384 2024-08-01T04:24:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.450+0000 7fc582dc2740 1 Processor -- start 2024-08-01T04:24:03.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.450+0000 7fc582dc2740 1 -- start start 2024-08-01T04:24:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27c0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c27a8f60 0x55e9c27c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc582dc2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781add40 con 0x55e9c27a8f60 2024-08-01T04:24:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc582dc2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ab610 con 0x55e9c279e6a0 2024-08-01T04:24:03.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc582dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781a9480 con 0x55e9c279dd80 2024-08-01T04:24:03.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27c0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57e200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c27a8f60 0x55e9c27c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57d800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:50898/0 (socket says 172.21.15.3:50898) 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57d800640 1 -- 172.21.15.3:0/2257718760 learned_addr learned my addr 172.21.15.3:0/2257718760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27c0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:48994/0 (socket says 172.21.15.3:48994) 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 -- 172.21.15.3:0/2257718760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 msgr2=0x55e9c27a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 -- 172.21.15.3:0/2257718760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c27a8f60 msgr2=0x55e9c27c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c27a8f60 0x55e9c27c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57ce00640 1 -- 172.21.15.3:0/2257718760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c27a4e00 con 0x55e9c279e6a0 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.451+0000 7fc57d800640 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.452+0000 7fc57ce00640 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27c0890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc564003e50 tx=0x7fc5640091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953e79ca9c6438aa server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.452+0000 7fc56a000640 1 -- 172.21.15.3:0/2257718760 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5640052c0 con 0x55e9c279e6a0 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.452+0000 7fc56a000640 1 -- 172.21.15.3:0/2257718760 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564021040 con 0x55e9c279e6a0 2024-08-01T04:24:03.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.452+0000 7fc56a000640 1 -- 172.21.15.3:0/2257718760 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56401c400 con 0x55e9c279e6a0 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 -- 172.21.15.3:0/2257718760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 msgr2=0x55e9c27c0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27c0890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc564003e50 tx=0x7fc5640091e0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 -- 172.21.15.3:0/2257718760 shutdown_connections 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c27a8f60 0x55e9c27c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c279e6a0 0x55e9c27c0890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 --2- 172.21.15.3:0/2257718760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 -- 172.21.15.3:0/2257718760 >> 172.21.15.3:0/2257718760 conn(0x55e9c279a310 msgr2=0x55e9c27a7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 -- 172.21.15.3:0/2257718760 shutdown_connections 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.453+0000 7fc582dc2740 1 -- 172.21.15.3:0/2257718760 wait complete. 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 -- start start 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 Processor -- start 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279e6a0 0x55e9c27acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781a9ad0 con 0x55e9c279e6a0 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781aaf40 con 0x55e9c27a8f60 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.454+0000 7fc582dc2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781aa1a0 con 0x55e9c279dd80 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc582dc2740 10 client.0 _ll_register_callbacks cb 0x7fc56c00c550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279e6a0 0x55e9c27acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279e6a0 0x55e9c27acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37526/0 (socket says 172.21.15.3:37526) 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57ce00640 1 -- 172.21.15.3:0/1640489289 learned_addr learned my addr 172.21.15.3:0/1640489289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57d800640 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 -- 172.21.15.3:0/1640489289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 msgr2=0x55e9c27b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 -- 172.21.15.3:0/1640489289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279e6a0 msgr2=0x55e9c27acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279e6a0 0x55e9c27acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.455+0000 7fc57e200640 1 -- 172.21.15.3:0/1640489289 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c279e4d0 con 0x55e9c27a8f60 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc57e200640 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ad090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc57002ae10 tx=0x7fc570030b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc343a2d2105106 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc55f400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc55f400640 1 -- 172.21.15.3:0/1640489289 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5700098a0 con 0x55e9c27a8f60 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc55f400640 1 -- 172.21.15.3:0/1640489289 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fc570004040 con 0x55e9c27a8f60 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc564021040 con 0x55e9c27a8f60 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc55f400640 1 -- 172.21.15.3:0/1640489289 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc570054050 con 0x55e9c27a8f60 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.456+0000 7fc582dc2740 10 client.26141 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:03.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9c28237b0 con 0x55e9c27a8f60 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc55f400640 1 -- 172.21.15.3:0/1640489289 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc564021040 con 0x55e9c27a8f60 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc55f400640 1 -- 172.21.15.3:0/1640489289 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc57004f030 con 0x55e9c27a8f60 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 10 client.26141 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 20 client.26141 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c2823f00 0x55e9c2826320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 4 client.26141 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:24:03.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.457+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9c270b670 con 0x55e9c2823f00 2024-08-01T04:24:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.458+0000 7fc57d800640 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c2823f00 0x55e9c2826320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.458+0000 7fc57d800640 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c2823f00 0x55e9c2826320 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884b459dea645105 server_cookie=47989476c614ddca in_seq=0 out_seq=0 2024-08-01T04:24:03.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.458+0000 7fc55f400640 10 client.26141 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.458+0000 7fc55f400640 1 -- 172.21.15.3:0/1640489289 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e9c270b670 con 0x55e9c2823f00 2024-08-01T04:24:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.458+0000 7fc55f400640 10 client.26141 handle_command_reply: tid=0 2024-08-01T04:24:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.458+0000 7fc582dc2740 1 client.26141 shutdown 2024-08-01T04:24:03.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 msgr2=0x55e9c27ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ad090 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc57002ae10 tx=0x7fc570030b60 comp rx=0 tx=0).stop 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 shutdown_connections 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9c2823f00 0x55e9c2826320 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9c27a8f60 0x55e9c27ad090 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9c279e6a0 0x55e9c27acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 --2- 172.21.15.3:0/1640489289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9c279dd80 0x55e9c27b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc55f400640 0 client.26141 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 >> 172.21.15.3:0/1640489289 conn(0x55e9c279a310 msgr2=0x55e9c27a76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 shutdown_connections 2024-08-01T04:24:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 1 -- 172.21.15.3:0/1640489289 wait complete. 2024-08-01T04:24:03.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.459+0000 7fc582dc2740 20 client.26141 trim_cache size 0 max 16384 2024-08-01T04:24:03.468 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:24:03.468 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:24:03.468 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout: "51e75c29-472d-47d4-80b3-a9600402b771": { 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "51e75c29-472d-47d4-80b3-a9600402b771", 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:24:03.469 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:24:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc5600a2d70 msgr2=0x7fc5600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc5600a2d70 0x7fc5600a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc5780ffa80 tx=0x7fc564012040 comp rx=0 tx=0).stop 2024-08-01T04:24:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 msgr2=0x7fc578197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:03.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 0x7fc578197fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00c600 tx=0x7fc56c00cac0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 shutdown_connections 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc5600a2d70 0x7fc5600a5230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5781076f0 0x7fc57819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc578102090 0x7fc578198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 --2- 172.21.15.3:0/1162199873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5781016f0 0x7fc578197fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 >> 172.21.15.3:0/1162199873 conn(0x7fc5780fd470 msgr2=0x7fc5780fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 shutdown_connections 2024-08-01T04:24:03.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:03.460+0000 7fc57ec00640 1 -- 172.21.15.3:0/1162199873 wait complete. 2024-08-01T04:24:03.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:03 smithi160 ceph-mon[37224]: pgmap v238: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 7.93k op/s 2024-08-01T04:24:04.011 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:03 smithi003 ceph-mon[32262]: pgmap v238: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 7.93k op/s 2024-08-01T04:24:04.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:03 smithi163 ceph-mon[37383]: pgmap v238: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 33 KiB/s rd, 42 MiB/s wr, 7.93k op/s 2024-08-01T04:24:04.555 INFO:tasks.cephfs.filesystem:scrub status for tag:51e75c29-472d-47d4-80b3-a9600402b771 - {'path': '/', 'tag': '51e75c29-472d-47d4-80b3-a9600402b771', 'options': 'recursive,force'} 2024-08-01T04:24:06.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:05 smithi163 ceph-mon[37383]: pgmap v239: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 6.37k op/s 2024-08-01T04:24:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:05 smithi003 ceph-mon[32262]: pgmap v239: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 6.37k op/s 2024-08-01T04:24:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:05 smithi160 ceph-mon[37224]: pgmap v239: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 33 KiB/s rd, 33 MiB/s wr, 6.37k op/s 2024-08-01T04:24:07.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:06 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:24:07.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:06 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:24:07.053 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:06 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:24:08.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:07 smithi163 ceph-mon[37383]: pgmap v240: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 52 KiB/s rd, 32 MiB/s wr, 6.36k op/s 2024-08-01T04:24:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:07 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:24:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:07 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:24:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:07 smithi163 ceph-mon[37383]: osdmap e94: 12 total, 12 up, 12 in 2024-08-01T04:24:08.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:07 smithi003 ceph-mon[32262]: pgmap v240: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 52 KiB/s rd, 32 MiB/s wr, 6.36k op/s 2024-08-01T04:24:08.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:07 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:24:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:07 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:24:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:07 smithi003 ceph-mon[32262]: osdmap e94: 12 total, 12 up, 12 in 2024-08-01T04:24:08.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:07 smithi160 ceph-mon[37224]: pgmap v240: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 52 KiB/s rd, 32 MiB/s wr, 6.36k op/s 2024-08-01T04:24:08.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:07 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:24:08.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:07 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:24:08.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:07 smithi160 ceph-mon[37224]: osdmap e94: 12 total, 12 up, 12 in 2024-08-01T04:24:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:09 smithi163 ceph-mon[37383]: pgmap v242: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 3.26k op/s 2024-08-01T04:24:10.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:09 smithi003 ceph-mon[32262]: pgmap v242: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 3.26k op/s 2024-08-01T04:24:10.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:09 smithi160 ceph-mon[37224]: pgmap v242: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 3.26k op/s 2024-08-01T04:24:12.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:11 smithi163 ceph-mon[37383]: pgmap v243: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.30k op/s 2024-08-01T04:24:12.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:11 smithi003 ceph-mon[32262]: pgmap v243: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.30k op/s 2024-08-01T04:24:12.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:11 smithi160 ceph-mon[37224]: pgmap v243: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 2.30k op/s 2024-08-01T04:24:13.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:12 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:24:13.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:12 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:24:13.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:12 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:24:14.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:13 smithi163 ceph-mon[37383]: pgmap v244: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 243 KiB/s wr, 66 op/s 2024-08-01T04:24:14.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:13 smithi003 ceph-mon[32262]: pgmap v244: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 243 KiB/s wr, 66 op/s 2024-08-01T04:24:14.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:13 smithi160 ceph-mon[37224]: pgmap v244: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 243 KiB/s wr, 66 op/s 2024-08-01T04:24:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:14 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:24:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:14 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:24:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:14 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:24:15.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:14 smithi163 ceph-mon[37383]: osdmap e95: 12 total, 12 up, 12 in 2024-08-01T04:24:15.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:14 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:24:15.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:14 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:24:15.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:14 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:24:15.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:14 smithi003 ceph-mon[32262]: osdmap e95: 12 total, 12 up, 12 in 2024-08-01T04:24:15.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:14 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:24:15.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:14 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:24:15.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:14 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:24:15.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:14 smithi160 ceph-mon[37224]: osdmap e95: 12 total, 12 up, 12 in 2024-08-01T04:24:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:15 smithi163 ceph-mon[37383]: pgmap v245: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 24 op/s 2024-08-01T04:24:16.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:15 smithi003 ceph-mon[32262]: pgmap v245: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 24 op/s 2024-08-01T04:24:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:15 smithi160 ceph-mon[37224]: pgmap v245: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 24 op/s 2024-08-01T04:24:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:17 smithi163 ceph-mon[37383]: pgmap v247: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-08-01T04:24:18.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:17 smithi003 ceph-mon[32262]: pgmap v247: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-08-01T04:24:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:17 smithi160 ceph-mon[37224]: pgmap v247: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-08-01T04:24:20.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:19 smithi163 ceph-mon[37383]: pgmap v248: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-08-01T04:24:20.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:19 smithi003 ceph-mon[32262]: pgmap v248: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-08-01T04:24:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:19 smithi160 ceph-mon[37224]: pgmap v248: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-08-01T04:24:22.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:21 smithi163 ceph-mon[37383]: pgmap v249: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:22.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:21 smithi003 ceph-mon[32262]: pgmap v249: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:22.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:21 smithi160 ceph-mon[37224]: pgmap v249: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:24.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:23 smithi163 ceph-mon[37383]: pgmap v250: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:23 smithi003 ceph-mon[32262]: pgmap v250: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:23 smithi160 ceph-mon[37224]: pgmap v250: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:26.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:25 smithi163 ceph-mon[37383]: pgmap v251: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:26.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:25 smithi003 ceph-mon[32262]: pgmap v251: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:26.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:25 smithi160 ceph-mon[37224]: pgmap v251: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2024-08-01T04:24:28.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:27 smithi163 ceph-mon[37383]: pgmap v252: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-08-01T04:24:28.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:27 smithi003 ceph-mon[32262]: pgmap v252: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-08-01T04:24:28.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:27 smithi160 ceph-mon[37224]: pgmap v252: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-08-01T04:24:30.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:29 smithi163 ceph-mon[37383]: pgmap v253: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-08-01T04:24:30.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:29 smithi003 ceph-mon[32262]: pgmap v253: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-08-01T04:24:30.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:29 smithi160 ceph-mon[37224]: pgmap v253: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-08-01T04:24:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:31 smithi163 ceph-mon[37383]: pgmap v254: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 5 op/s 2024-08-01T04:24:32.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:31 smithi003 ceph-mon[32262]: pgmap v254: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 5 op/s 2024-08-01T04:24:32.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:31 smithi160 ceph-mon[37224]: pgmap v254: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.8 KiB/s wr, 5 op/s 2024-08-01T04:24:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:33 smithi163 ceph-mon[37383]: pgmap v255: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:24:34.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:33 smithi003 ceph-mon[32262]: pgmap v255: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:24:34.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:33 smithi160 ceph-mon[37224]: pgmap v255: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:24:34.556 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:24:35.039 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:35.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.728+0000 7f6800e00640 1 -- 172.21.15.3:0/4214155027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc1016f0 msgr2=0x7f67fc101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.728+0000 7f6800e00640 1 --2- 172.21.15.3:0/4214155027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc1016f0 0x7f67fc101b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009860 tx=0x7f67e402a340 comp rx=0 tx=0).stop 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 -- 172.21.15.3:0/4214155027 shutdown_connections 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 --2- 172.21.15.3:0/4214155027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67fc102090 0x7f67fc10e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 --2- 172.21.15.3:0/4214155027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc1016f0 0x7f67fc101b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 --2- 172.21.15.3:0/4214155027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1076f0 0x7f67fc107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 -- 172.21.15.3:0/4214155027 >> 172.21.15.3:0/4214155027 conn(0x7f67fc0fd470 msgr2=0x7f67fc0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 -- 172.21.15.3:0/4214155027 shutdown_connections 2024-08-01T04:24:35.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.729+0000 7f6800e00640 1 -- 172.21.15.3:0/4214155027 wait complete. 2024-08-01T04:24:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.730+0000 7f6800e00640 1 Processor -- start 2024-08-01T04:24:35.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.730+0000 7f6800e00640 1 -- start start 2024-08-01T04:24:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f6800e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1016f0 0x7f67fc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f6800e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 0x7f67fc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f6800e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67fc1076f0 0x7f67fc19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f6800e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc111af0 con 0x7f67fc1016f0 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f6800e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc111970 con 0x7f67fc102090 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f6800e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc111c70 con 0x7f67fc1076f0 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f67fb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1016f0 0x7f67fc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 0x7f67fc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f67fb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1016f0 0x7f67fc19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60750/0 (socket says 172.21.15.3:60750) 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 0x7f67fc19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50116/0 (socket says 172.21.15.3:50116) 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f67fb400640 1 -- 172.21.15.3:0/4056360577 learned_addr learned my addr 172.21.15.3:0/4056360577 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:35.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.731+0000 7f67fbe00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67fc1076f0 0x7f67fc19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.732+0000 7f67f2a00640 1 -- 172.21.15.3:0/4056360577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67fc1076f0 msgr2=0x7f67fc19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.732+0000 7f67f2a00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67fc1076f0 0x7f67fc19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.732+0000 7f67f2a00640 1 -- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1016f0 msgr2=0x7f67fc19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.732+0000 7f67f2a00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1016f0 0x7f67fc19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.732+0000 7f67f2a00640 1 -- 172.21.15.3:0/4056360577 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67fc19f150 con 0x7f67fc102090 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.732+0000 7f67f2a00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 0x7f67fc19a6e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67e402a860 tx=0x7f67e4033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.733+0000 7f67f8c00640 1 -- 172.21.15.3:0/4056360577 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4037040 con 0x7f67fc102090 2024-08-01T04:24:35.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.733+0000 7f67f8c00640 1 -- 172.21.15.3:0/4056360577 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e4033d50 con 0x7f67fc102090 2024-08-01T04:24:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.733+0000 7f67f8c00640 1 -- 172.21.15.3:0/4056360577 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40039e0 con 0x7f67fc102090 2024-08-01T04:24:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.733+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67fc19f3e0 con 0x7f67fc102090 2024-08-01T04:24:35.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.733+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67fc1a6d20 con 0x7f67fc102090 2024-08-01T04:24:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 Processor -- start 2024-08-01T04:24:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f67f8c00640 1 -- 172.21.15.3:0/4056360577 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f67e4042020 con 0x7f67fc102090 2024-08-01T04:24:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 -- start start 2024-08-01T04:24:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f13160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f1df60 0x55c565f20350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c565e78bf0 con 0x55c565f136a0 2024-08-01T04:24:35.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c565e10bd0 con 0x55c565f1df60 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.737+0000 7f6804ff0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c565e74bb0 con 0x55c565f12d80 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f13160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60754/0 (socket says 172.21.15.3:60754) 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f1df60 0x55c565f20350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fb400640 1 -- 172.21.15.3:0/2659322474 learned_addr learned my addr 172.21.15.3:0/2659322474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fbe00640 1 -- 172.21.15.3:0/2659322474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f1df60 msgr2=0x55c565f20350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fbe00640 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f1df60 0x55c565f20350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fbe00640 1 -- 172.21.15.3:0/2659322474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 msgr2=0x55c565f1da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fbe00640 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fbe00640 1 -- 172.21.15.3:0/2659322474 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c565e178a0 con 0x55c565f12d80 2024-08-01T04:24:35.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.738+0000 7f67fb400640 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:35.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.739+0000 7f67fbe00640 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f13160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009b80 tx=0x7f67f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbfa6740793baebf server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.739+0000 7f67f0c00640 1 -- 172.21.15.3:0/2659322474 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004050 con 0x55c565f12d80 2024-08-01T04:24:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.739+0000 7f67f0c00640 1 -- 172.21.15.3:0/2659322474 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4005500 con 0x55c565f12d80 2024-08-01T04:24:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.739+0000 7f67f0c00640 1 -- 172.21.15.3:0/2659322474 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004e00 con 0x55c565f12d80 2024-08-01T04:24:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.739+0000 7f67f8c00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f67e00a2cf0 0x7f67e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f67f8c00640 1 -- 172.21.15.3:0/4056360577 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f67e40ee9a0 con 0x7f67fc102090 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 -- 172.21.15.3:0/2659322474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 msgr2=0x55c565f13160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f13160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009b80 tx=0x7f67f4011810 comp rx=0 tx=0).stop 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f67fb400640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f67e00a2cf0 0x7f67e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 -- 172.21.15.3:0/2659322474 shutdown_connections 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f1df60 0x55c565f20350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 --2- 172.21.15.3:0/2659322474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f13160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 -- 172.21.15.3:0/2659322474 >> 172.21.15.3:0/2659322474 conn(0x55c565f0f310 msgr2=0x55c565f0f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.740+0000 7f6804ff0740 1 -- 172.21.15.3:0/2659322474 shutdown_connections 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.741+0000 7f6804ff0740 1 -- 172.21.15.3:0/2659322474 wait complete. 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.741+0000 7f67fb400640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f67e00a2cf0 0x7f67e00a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55c565f10170 tx=0x7f67ec010040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.741+0000 7f6804ff0740 1 -- start start 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.741+0000 7f6804ff0740 1 Processor -- start 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f1ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 0x55c565f1b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f1b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c565f20b30 con 0x55c565f1df60 2024-08-01T04:24:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c565e33d40 con 0x55c565f12d80 2024-08-01T04:24:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c565f20cb0 con 0x55c565f136a0 2024-08-01T04:24:35.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f6804ff0740 10 client.0 _ll_register_callbacks cb 0x55c565dfcb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 0x55c565f1b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 0x55c565f1b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:48568/0 (socket says 172.21.15.3:48568) 2024-08-01T04:24:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f67f2a00640 1 -- 172.21.15.3:0/592817651 learned_addr learned my addr 172.21.15.3:0/592817651 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:35.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f67fbe00640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f1b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.742+0000 7f67fb400640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f1ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67f2a00640 1 -- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 msgr2=0x55c565f1ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67f2a00640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f1ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67f2a00640 1 -- 172.21.15.3:0/592817651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 msgr2=0x55c565f1b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67f2a00640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f1b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67f2a00640 1 -- 172.21.15.3:0/592817651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c565bea3c0 con 0x55c565f136a0 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67f2a00640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 0x55c565f1b390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005e00 tx=0x7f67e4004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba46f47fa7a2f944 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67e9600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:24:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.743+0000 7f67e9600640 1 -- 172.21.15.3:0/592817651 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40baa40 con 0x55c565f136a0 2024-08-01T04:24:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f4005500 con 0x55c565f136a0 2024-08-01T04:24:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f67e9600640 1 -- 172.21.15.3:0/592817651 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f67e40c36e0 con 0x55c565f136a0 2024-08-01T04:24:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f67e9600640 1 -- 172.21.15.3:0/592817651 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67e40ba860 con 0x55c565f136a0 2024-08-01T04:24:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f6804ff0740 10 client.26128 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:35.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c565f20b80 con 0x55c565f136a0 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f67e9600640 1 -- 172.21.15.3:0/592817651 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4005500 con 0x55c565f136a0 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f67e9600640 1 -- 172.21.15.3:0/592817651 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f67e403a070 con 0x55c565f136a0 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f6804ff0740 10 client.26128 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.744+0000 7f6804ff0740 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.745+0000 7f6804ff0740 20 client.26128 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.745+0000 7f6804ff0740 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f2dbe0 0x55c565f30020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.745+0000 7f6804ff0740 4 client.26128 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:35.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.745+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c565e86160 con 0x55c565f2dbe0 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.745+0000 7f67fb400640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f2dbe0 0x55c565f30020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.746+0000 7f67fb400640 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f2dbe0 0x55c565f30020 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f0cf3eb3ab45d5 server_cookie=fc537c4a28bfe1ff in_seq=0 out_seq=0 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.746+0000 7f67e9600640 10 client.26128 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f67e9600640 1 -- 172.21.15.3:0/592817651 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c565e86160 con 0x55c565f2dbe0 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f67e9600640 10 client.26128 handle_command_reply: tid=0 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 client.26128 shutdown 2024-08-01T04:24:35.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 msgr2=0x55c565f1b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 0x55c565f1b390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005e00 tx=0x7f67e4004510 comp rx=0 tx=0).stop 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 shutdown_connections 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f2dbe0 0x55c565f30020 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f1b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f136a0 0x55c565f1b390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 --2- 172.21.15.3:0/592817651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f1ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 >> 172.21.15.3:0/592817651 conn(0x55c565f0f310 msgr2=0x55c565f15040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f67e9600640 0 client.26128 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 shutdown_connections 2024-08-01T04:24:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.747+0000 7f6804ff0740 1 -- 172.21.15.3:0/592817651 wait complete. 2024-08-01T04:24:35.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.748+0000 7f6804ff0740 20 client.26128 trim_cache size 0 max 16384 2024-08-01T04:24:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 Processor -- start 2024-08-01T04:24:35.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 -- start start 2024-08-01T04:24:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f1df60 0x55c565f1ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1add50 con 0x55c565f136a0 2024-08-01T04:24:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1ab620 con 0x55c565f12d80 2024-08-01T04:24:35.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.767+0000 7f6804ff0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1a94f0 con 0x55c565f1df60 2024-08-01T04:24:35.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67f2a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60776/0 (socket says 172.21.15.3:60776) 2024-08-01T04:24:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67f2a00640 1 -- 172.21.15.3:0/34011958 learned_addr learned my addr 172.21.15.3:0/34011958 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fbe00640 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f1df60 0x55c565f1ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fb400640 1 -- 172.21.15.3:0/34011958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f1df60 msgr2=0x55c565f1ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fb400640 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f1df60 0x55c565f1ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fb400640 1 -- 172.21.15.3:0/34011958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 msgr2=0x55c565f1b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fb400640 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.768+0000 7f67fb400640 1 -- 172.21.15.3:0/34011958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c565f12bb0 con 0x55c565f12d80 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.769+0000 7f67fb400640 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f157e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55c565f30830 tx=0x7f67ec016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1058ab57994fa036 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.769+0000 7f67ebe00640 1 -- 172.21.15.3:0/34011958 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec020040 con 0x55c565f12d80 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.769+0000 7f67ebe00640 1 -- 172.21.15.3:0/34011958 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec034070 con 0x55c565f12d80 2024-08-01T04:24:35.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.769+0000 7f67ebe00640 1 -- 172.21.15.3:0/34011958 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec018710 con 0x55c565f12d80 2024-08-01T04:24:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 -- 172.21.15.3:0/34011958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 msgr2=0x55c565f157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f157e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55c565f30830 tx=0x7f67ec016000 comp rx=0 tx=0).stop 2024-08-01T04:24:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 -- 172.21.15.3:0/34011958 shutdown_connections 2024-08-01T04:24:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f1df60 0x55c565f1ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f136a0 0x55c565f1b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 --2- 172.21.15.3:0/34011958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f12d80 0x55c565f157e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.770+0000 7f6804ff0740 1 -- 172.21.15.3:0/34011958 >> 172.21.15.3:0/34011958 conn(0x55c565f0f310 msgr2=0x55c565f1ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.771+0000 7f6804ff0740 1 -- 172.21.15.3:0/34011958 shutdown_connections 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.771+0000 7f6804ff0740 1 -- 172.21.15.3:0/34011958 wait complete. 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.771+0000 7f6804ff0740 1 -- start start 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.771+0000 7f6804ff0740 1 Processor -- start 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f26500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f136a0 0x55c565f21830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f21d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1a9b40 con 0x55c565f1df60 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1aaf50 con 0x55c565f136a0 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1aa1b0 con 0x55c565f12d80 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f6804ff0740 10 client.0 _ll_register_callbacks cb 0x7f67e40fca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f67fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f26500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f67fb400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f26500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:48592/0 (socket says 172.21.15.3:48592) 2024-08-01T04:24:35.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f67fb400640 1 -- 172.21.15.3:0/794292983 learned_addr learned my addr 172.21.15.3:0/794292983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f67f2a00640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f136a0 0x55c565f21830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.772+0000 7f67fbe00640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f21d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67fb400640 1 -- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f136a0 msgr2=0x55c565f21830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67fb400640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f136a0 0x55c565f21830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67fb400640 1 -- 172.21.15.3:0/794292983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 msgr2=0x55c565f21d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67fb400640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f21d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67fb400640 1 -- 172.21.15.3:0/794292983 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c565f22410 con 0x55c565f12d80 2024-08-01T04:24:35.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67fb400640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f26500 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67ec027e20 tx=0x7f67ec027e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49004ae9c06c7579 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:35.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.773+0000 7f67e9600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:24:35.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.774+0000 7f67e9600640 1 -- 172.21.15.3:0/794292983 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec03c030 con 0x55c565f12d80 2024-08-01T04:24:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.774+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ec034070 con 0x55c565f12d80 2024-08-01T04:24:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.774+0000 7f67e9600640 1 -- 172.21.15.3:0/794292983 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f67ec004c50 con 0x55c565f12d80 2024-08-01T04:24:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.774+0000 7f67e9600640 1 -- 172.21.15.3:0/794292983 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67ec046050 con 0x55c565f12d80 2024-08-01T04:24:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.774+0000 7f6804ff0740 10 client.26134 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.774+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c565f98740 con 0x55c565f12d80 2024-08-01T04:24:35.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f67e9600640 1 -- 172.21.15.3:0/794292983 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ec034070 con 0x55c565f12d80 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f67e9600640 1 -- 172.21.15.3:0/794292983 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f67ec00b040 con 0x55c565f12d80 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f6804ff0740 10 client.26134 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f6804ff0740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f6804ff0740 20 client.26134 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f6804ff0740 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f98e00 0x55c565f9b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f6804ff0740 4 client.26134 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c565e80670 con 0x55c565f98e00 2024-08-01T04:24:35.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.775+0000 7f67f2a00640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f98e00 0x55c565f9b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f67f2a00640 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f98e00 0x55c565f9b220 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74bce4bc67d4f2d server_cookie=1622d12437167a69 in_seq=0 out_seq=0 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f67e9600640 10 client.26134 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f67e9600640 1 -- 172.21.15.3:0/794292983 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c565e80670 con 0x55c565f98e00 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f67e9600640 10 client.26134 handle_command_reply: tid=0 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f6804ff0740 1 client.26134 shutdown 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 msgr2=0x55c565f26500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.776+0000 7f6804ff0740 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f26500 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67ec027e20 tx=0x7f67ec027e80 comp rx=0 tx=0).stop 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 shutdown_connections 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c565f98e00 0x55c565f9b220 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c565f1df60 0x55c565f21d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c565f136a0 0x55c565f21830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 --2- 172.21.15.3:0/794292983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c565f12d80 0x55c565f26500 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f67e9600640 0 client.26134 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 >> 172.21.15.3:0/794292983 conn(0x55c565f0f310 msgr2=0x55c565f0e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 shutdown_connections 2024-08-01T04:24:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 1 -- 172.21.15.3:0/794292983 wait complete. 2024-08-01T04:24:35.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.777+0000 7f6804ff0740 20 client.26134 trim_cache size 0 max 16384 2024-08-01T04:24:35.784 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:24:35.784 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:24:35.785 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:24:35.785 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:24:35.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f67e00a2cf0 msgr2=0x7f67e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f67e00a2cf0 0x7f67e00a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55c565f10170 tx=0x7f67ec010040 comp rx=0 tx=0).stop 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 msgr2=0x7f67fc19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 0x7f67fc19a6e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67e402a860 tx=0x7f67e4033b60 comp rx=0 tx=0).stop 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 shutdown_connections 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f67e00a2cf0 0x7f67e00a51b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67fc1076f0 0x7f67fc19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67fc102090 0x7f67fc19a6e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 --2- 172.21.15.3:0/4056360577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67fc1016f0 0x7f67fc19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 >> 172.21.15.3:0/4056360577 conn(0x7f67fc0fd470 msgr2=0x7f67fc10ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 shutdown_connections 2024-08-01T04:24:35.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:35.778+0000 7f6800e00640 1 -- 172.21.15.3:0/4056360577 wait complete. 2024-08-01T04:24:36.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:35 smithi163 ceph-mon[37383]: pgmap v256: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:24:36.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:35 smithi003 ceph-mon[32262]: pgmap v256: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:24:36.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:35 smithi160 ceph-mon[37224]: pgmap v256: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:24:36.777 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:24:36.778 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:24:37.241 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:37.881 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:37 smithi003 ceph-mon[32262]: pgmap v257: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:24:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.886+0000 7fc623200640 1 -- 172.21.15.3:0/383894714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c10d260 msgr2=0x7fc61c114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.886+0000 7fc623200640 1 --2- 172.21.15.3:0/383894714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c10d260 0x7fc61c114de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc614009ca0 tx=0x7fc61402a8c0 comp rx=0 tx=0).stop 2024-08-01T04:24:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 -- 172.21.15.3:0/383894714 shutdown_connections 2024-08-01T04:24:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 --2- 172.21.15.3:0/383894714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c10d260 0x7fc61c114de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:37.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 --2- 172.21.15.3:0/383894714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108b30 0x7fc61c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 --2- 172.21.15.3:0/383894714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc61c108180 0x7fc61c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 -- 172.21.15.3:0/383894714 >> 172.21.15.3:0/383894714 conn(0x7fc61c0778d0 msgr2=0x7fc61c079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 -- 172.21.15.3:0/383894714 shutdown_connections 2024-08-01T04:24:37.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.887+0000 7fc623200640 1 -- 172.21.15.3:0/383894714 wait complete. 2024-08-01T04:24:37.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.888+0000 7fc623200640 1 Processor -- start 2024-08-01T04:24:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.888+0000 7fc623200640 1 -- start start 2024-08-01T04:24:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.888+0000 7fc623200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 0x7fc61c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.888+0000 7fc623200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 0x7fc61c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc623200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc61c10d260 0x7fc61c1a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc623200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c1182e0 con 0x7fc61c108b30 2024-08-01T04:24:37.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc623200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c118160 con 0x7fc61c10d260 2024-08-01T04:24:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc623200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c118460 con 0x7fc61c108180 2024-08-01T04:24:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc613e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 0x7fc61c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 0x7fc61c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc613e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 0x7fc61c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60800/0 (socket says 172.21.15.3:60800) 2024-08-01T04:24:37.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 0x7fc61c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:48606/0 (socket says 172.21.15.3:48606) 2024-08-01T04:24:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc613e00640 1 -- 172.21.15.3:0/655701515 learned_addr learned my addr 172.21.15.3:0/655701515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc621200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc61c10d260 0x7fc61c1a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 -- 172.21.15.3:0/655701515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc61c10d260 msgr2=0x7fc61c1a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc61c10d260 0x7fc61c1a2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 -- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 msgr2=0x7fc61c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:37.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 0x7fc61c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc61be00640 1 -- 172.21.15.3:0/655701515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c1a36d0 con 0x7fc61c108180 2024-08-01T04:24:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.889+0000 7fc613e00640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 0x7fc61c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.890+0000 7fc61be00640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 0x7fc61c19e720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00e9f0 tx=0x7fc60c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.890+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00ccc0 con 0x7fc61c108180 2024-08-01T04:24:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.890+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c1a39c0 con 0x7fc61c108180 2024-08-01T04:24:37.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.890+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc60c010910 con 0x7fc61c108180 2024-08-01T04:24:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.890+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c018960 con 0x7fc61c108180 2024-08-01T04:24:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.890+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c1ab2a0 con 0x7fc61c108180 2024-08-01T04:24:37.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.892+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c108560 con 0x7fc61c108180 2024-08-01T04:24:37.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.893+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc60c010ac0 con 0x7fc61c108180 2024-08-01T04:24:37.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.895+0000 7fc619600640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc6000a2d70 0x7fc6000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:37.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.895+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fc60c0cfb60 con 0x7fc61c108180 2024-08-01T04:24:37.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.895+0000 7fc613e00640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc6000a2d70 0x7fc6000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:37.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.896+0000 7fc613e00640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc6000a2d70 0x7fc6000a5230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc61c107c80 tx=0x7fc604005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:37.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:37.897+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc60c09c060 con 0x7fc61c108180 2024-08-01T04:24:38.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.121+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc61c1ab4d0 con 0x7fc61c108180 2024-08-01T04:24:38.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.122+0000 7fc619600640 1 -- 172.21.15.3:0/655701515 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7fc60c0a0360 con 0x7fc61c108180 2024-08-01T04:24:38.123 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:24:38.124 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:24:38.124 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:24:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc6000a2d70 msgr2=0x7fc6000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc6000a2d70 0x7fc6000a5230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc61c107c80 tx=0x7fc604005f90 comp rx=0 tx=0).stop 2024-08-01T04:24:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 msgr2=0x7fc61c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 0x7fc61c19e720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00e9f0 tx=0x7fc60c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:24:38.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 shutdown_connections 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc61c10d260 0x7fc61c1a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c108b30 0x7fc61c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fc6000a2d70 0x7fc6000a5230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 --2- 172.21.15.3:0/655701515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61c108180 0x7fc61c19e720 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 >> 172.21.15.3:0/655701515 conn(0x7fc61c0778d0 msgr2=0x7fc61c079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 shutdown_connections 2024-08-01T04:24:38.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:38.125+0000 7fc623200640 1 -- 172.21.15.3:0/655701515 wait complete. 2024-08-01T04:24:38.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:37 smithi160 ceph-mon[37224]: pgmap v257: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:24:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:37 smithi163 ceph-mon[37383]: pgmap v257: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-08-01T04:24:38.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:24:39.062 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:38 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/655701515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:24:39.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:38 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/655701515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:24:39.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:38 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/655701515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:24:39.463 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:39.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:39 smithi003 ceph-mon[32262]: pgmap v258: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:24:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:39 smithi160 ceph-mon[37224]: pgmap v258: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 -- 172.21.15.3:0/3749247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076d70 msgr2=0x7fb508112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 --2- 172.21.15.3:0/3749247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076d70 0x7fb508112060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009860 tx=0x7fb4f402a340 comp rx=0 tx=0).stop 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 -- 172.21.15.3:0/3749247824 shutdown_connections 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 --2- 172.21.15.3:0/3749247824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5081125a0 0x7fb508114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 --2- 172.21.15.3:0/3749247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076d70 0x7fb508112060 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 --2- 172.21.15.3:0/3749247824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076450 0x7fb508076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.155+0000 7fb50f000640 1 -- 172.21.15.3:0/3749247824 >> 172.21.15.3:0/3749247824 conn(0x7fb5080ff600 msgr2=0x7fb508101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:40.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.156+0000 7fb50f000640 1 -- 172.21.15.3:0/3749247824 shutdown_connections 2024-08-01T04:24:40.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.156+0000 7fb50f000640 1 -- 172.21.15.3:0/3749247824 wait complete. 2024-08-01T04:24:40.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 Processor -- start 2024-08-01T04:24:40.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 -- start start 2024-08-01T04:24:40.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 0x7fb50819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076d70 0x7fb50819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5081125a0 0x7fb5081a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb508118100 con 0x7fb508076d70 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb508117f80 con 0x7fb508076450 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.157+0000 7fb50f000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb508118280 con 0x7fb5081125a0 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 0x7fb50819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076d70 0x7fb50819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 0x7fb50819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50182/0 (socket says 172.21.15.3:50182) 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 -- 172.21.15.3:0/1521422158 learned_addr learned my addr 172.21.15.3:0/1521422158 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:40.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50e600640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5081125a0 0x7fb5081a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 -- 172.21.15.3:0/1521422158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5081125a0 msgr2=0x7fb5081a2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5081125a0 0x7fb5081a2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 -- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076d70 msgr2=0x7fb50819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076d70 0x7fb50819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50dc00640 1 -- 172.21.15.3:0/1521422158 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5081a35b0 con 0x7fb508076450 2024-08-01T04:24:40.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.158+0000 7fb50d200640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076d70 0x7fb50819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:40.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.159+0000 7fb50dc00640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 0x7fb50819e600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00e9f0 tx=0x7fb4fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:40.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.159+0000 7fb506000640 1 -- 172.21.15.3:0/1521422158 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00cd80 con 0x7fb508076450 2024-08-01T04:24:40.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.159+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5081a38a0 con 0x7fb508076450 2024-08-01T04:24:40.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.159+0000 7fb506000640 1 -- 172.21.15.3:0/1521422158 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc010910 con 0x7fb508076450 2024-08-01T04:24:40.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.159+0000 7fb506000640 1 -- 172.21.15.3:0/1521422158 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0188f0 con 0x7fb508076450 2024-08-01T04:24:40.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.159+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5081ab180 con 0x7fb508076450 2024-08-01T04:24:40.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.163+0000 7fb506000640 1 -- 172.21.15.3:0/1521422158 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb4fc010ac0 con 0x7fb508076450 2024-08-01T04:24:40.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.163+0000 7fb513086740 1 Processor -- start 2024-08-01T04:24:40.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.163+0000 7fb513086740 1 -- start start 2024-08-01T04:24:40.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb513086740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e3eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb513086740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e49380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb513086740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e4bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb513086740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557513da3bf0 con 0x557513e3e6e0 2024-08-01T04:24:40.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb513086740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557513d3bb20 con 0x557513e498c0 2024-08-01T04:24:40.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb513086740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557513d9fbb0 con 0x557513e3f000 2024-08-01T04:24:40.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb50e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e3eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb50e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e3eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60820/0 (socket says 172.21.15.3:60820) 2024-08-01T04:24:40.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb50e600640 1 -- 172.21.15.3:0/299129399 learned_addr learned my addr 172.21.15.3:0/299129399 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:40.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb50dc00640 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e49380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.164+0000 7fb50d200640 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e4bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50e600640 1 -- 172.21.15.3:0/299129399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 msgr2=0x557513e49380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50e600640 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e49380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50e600640 1 -- 172.21.15.3:0/299129399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 msgr2=0x557513e4bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50e600640 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e4bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50e600640 1 -- 172.21.15.3:0/299129399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557513d427f0 con 0x557513e3e6e0 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb506000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4f00a2d70 0x7fb4f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50e600640 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e3eac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb500009fc0 tx=0x7fb5000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3313849bc080e0c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb506000640 1 -- 172.21.15.3:0/1521422158 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fb4fc0cf860 con 0x7fb508076450 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.165+0000 7fb50dc00640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4f00a2d70 0x7fb4f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.166+0000 7fb4faa00640 1 -- 172.21.15.3:0/299129399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500004b20 con 0x557513e3e6e0 2024-08-01T04:24:40.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.166+0000 7fb4faa00640 1 -- 172.21.15.3:0/299129399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500004cc0 con 0x557513e3e6e0 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.166+0000 7fb4faa00640 1 -- 172.21.15.3:0/299129399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500005610 con 0x557513e3e6e0 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.166+0000 7fb513086740 1 -- 172.21.15.3:0/299129399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 msgr2=0x557513e3eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.166+0000 7fb50dc00640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4f00a2d70 0x7fb4f00a5230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x557513e3bad0 tx=0x7fb4fc0d5530 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.166+0000 7fb513086740 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e3eac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb500009fc0 tx=0x7fb5000095c0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 -- 172.21.15.3:0/299129399 shutdown_connections 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e4bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e49380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 --2- 172.21.15.3:0/299129399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e3eac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 -- 172.21.15.3:0/299129399 >> 172.21.15.3:0/299129399 conn(0x557513e3ac70 msgr2=0x557513e3b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:40.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 -- 172.21.15.3:0/299129399 shutdown_connections 2024-08-01T04:24:40.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 -- 172.21.15.3:0/299129399 wait complete. 2024-08-01T04:24:40.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.167+0000 7fb513086740 1 -- start start 2024-08-01T04:24:40.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 Processor -- start 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 0x557513e467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e46d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513e47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557513e4c490 con 0x557513e498c0 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557513d5ecf0 con 0x557513e3e6e0 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557513e4c610 con 0x557513e3f000 2024-08-01T04:24:40.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb513086740 10 client.0 _ll_register_callbacks cb 0x557513d4ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb50e600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 0x557513e467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb50e600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 0x557513e467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50200/0 (socket says 172.21.15.3:50200) 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.168+0000 7fb50e600640 1 -- 172.21.15.3:0/855108675 learned_addr learned my addr 172.21.15.3:0/855108675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50d200640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e46d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513e47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 -- 172.21.15.3:0/855108675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 msgr2=0x557513e46d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e46d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 -- 172.21.15.3:0/855108675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 msgr2=0x557513e47240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513e47240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 -- 172.21.15.3:0/855108675 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557513b153c0 con 0x557513e3e6e0 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50d200640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e46d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb50e600640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 0x557513e467c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb500004c80 tx=0x7fb5000051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10dc79fbb261c00 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.169+0000 7fb4ef400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb4ef400640 1 -- 172.21.15.3:0/855108675 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500005870 con 0x557513e3e6e0 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb500004cc0 con 0x557513e3e6e0 2024-08-01T04:24:40.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb4ef400640 1 -- 172.21.15.3:0/855108675 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fb500003970 con 0x557513e3e6e0 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb4ef400640 1 -- 172.21.15.3:0/855108675 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb500005a10 con 0x557513e3e6e0 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb513086740 10 client.26177 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557513e58150 con 0x557513e3e6e0 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.170+0000 7fb4ef400640 1 -- 172.21.15.3:0/855108675 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500004cc0 con 0x557513e3e6e0 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb4ef400640 1 -- 172.21.15.3:0/855108675 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fb50000b040 con 0x557513e3e6e0 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb513086740 10 client.26177 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb513086740 10 client.26177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:24:40.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb513086740 20 client.26177 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb513086740 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513e59500 0x557513e5b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb513086740 4 client.26177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557513db1160 con 0x557513e59500 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.171+0000 7fb50d200640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513e59500 0x557513e5b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.172+0000 7fb50d200640 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513e59500 0x557513e5b8f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d20acd843e6369e server_cookie=b86d7e60d754f568 in_seq=0 out_seq=0 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.172+0000 7fb4ef400640 10 client.26177 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.172+0000 7fb4ef400640 1 -- 172.21.15.3:0/855108675 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557513db1160 con 0x557513e59500 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.172+0000 7fb4ef400640 10 client.26177 handle_command_reply: tid=0 2024-08-01T04:24:40.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 client.26177 shutdown 2024-08-01T04:24:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 msgr2=0x557513e467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 0x557513e467c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb500004c80 tx=0x7fb5000051f0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 shutdown_connections 2024-08-01T04:24:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513e47240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513e59500 0x557513e5b8f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e46d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 --2- 172.21.15.3:0/855108675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3e6e0 0x557513e467c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 >> 172.21.15.3:0/855108675 conn(0x557513e3ac70 msgr2=0x557513e409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb4ef400640 0 client.26177 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:24:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 shutdown_connections 2024-08-01T04:24:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.173+0000 7fb513086740 1 -- 172.21.15.3:0/855108675 wait complete. 2024-08-01T04:24:40.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.174+0000 7fb513086740 20 client.26177 trim_cache size 0 max 16384 2024-08-01T04:24:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.188+0000 7fb513086740 1 Processor -- start 2024-08-01T04:24:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 -- start start 2024-08-01T04:24:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e47060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e5e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081b6510 con 0x557513e3e6e0 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081b3de0 con 0x557513e498c0 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb513086740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081b1c50 con 0x557513e3f000 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb50e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb50d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e47060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb50e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e45090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60826/0 (socket says 172.21.15.3:60826) 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.189+0000 7fb50e600640 1 -- 172.21.15.3:0/803013536 learned_addr learned my addr 172.21.15.3:0/803013536 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50e600640 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e5e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50d200640 1 -- 172.21.15.3:0/803013536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 msgr2=0x557513e5e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50d200640 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e5e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50d200640 1 -- 172.21.15.3:0/803013536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 msgr2=0x557513e45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50d200640 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e45090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50d200640 1 -- 172.21.15.3:0/803013536 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557513e4c510 con 0x557513e3f000 2024-08-01T04:24:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50e600640 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb50d200640 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e47060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4035040 tx=0x7fb4f4030460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fbd198d70bf3acc server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.190+0000 7fb4fa000640 1 -- 172.21.15.3:0/803013536 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4037040 con 0x557513e3f000 2024-08-01T04:24:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb4fa000640 1 -- 172.21.15.3:0/803013536 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f40035f0 con 0x557513e3f000 2024-08-01T04:24:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb4fa000640 1 -- 172.21.15.3:0/803013536 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40326a0 con 0x557513e3f000 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 -- 172.21.15.3:0/803013536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 msgr2=0x557513e47060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e47060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4035040 tx=0x7fb4f4030460 comp rx=0 tx=0).stop 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 -- 172.21.15.3:0/803013536 shutdown_connections 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e498c0 0x557513e5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3f000 0x557513e47060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 --2- 172.21.15.3:0/803013536 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e3e6e0 0x557513e45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 -- 172.21.15.3:0/803013536 >> 172.21.15.3:0/803013536 conn(0x557513e3ac70 msgr2=0x557513e48840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.191+0000 7fb513086740 1 -- 172.21.15.3:0/803013536 shutdown_connections 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.192+0000 7fb513086740 1 -- 172.21.15.3:0/803013536 wait complete. 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.192+0000 7fb513086740 1 -- start start 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.192+0000 7fb513086740 1 Processor -- start 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3e6e0 0x557513e5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3f000 0x557513e5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513ec1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081b22a0 con 0x557513e498c0 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081b3710 con 0x557513e3f000 2024-08-01T04:24:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5081b2970 con 0x557513e3e6e0 2024-08-01T04:24:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb513086740 10 client.0 _ll_register_callbacks cb 0x7fb500004bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb50e600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3e6e0 0x557513e5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb50e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513ec1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb50d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3f000 0x557513e5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb50e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513ec1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60840/0 (socket says 172.21.15.3:60840) 2024-08-01T04:24:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.193+0000 7fb50e600640 1 -- 172.21.15.3:0/2710496066 learned_addr learned my addr 172.21.15.3:0/2710496066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb50d200640 1 -- 172.21.15.3:0/2710496066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3e6e0 msgr2=0x557513e5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb50d200640 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3e6e0 0x557513e5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb50d200640 1 -- 172.21.15.3:0/2710496066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 msgr2=0x557513ec1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb50d200640 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513ec1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb50d200640 1 -- 172.21.15.3:0/2710496066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557513ec2470 con 0x557513e3f000 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb50d200640 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3f000 0x557513e5c3d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009780 tx=0x7fb4f40045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ab58d147fc4eeb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb4ef400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb4ef400640 1 -- 172.21.15.3:0/2710496066 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4053060 con 0x557513e3f000 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb4ef400640 1 -- 172.21.15.3:0/2710496066 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fb4f4030790 con 0x557513e3f000 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.194+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f40035f0 con 0x557513e3f000 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb4ef400640 1 -- 172.21.15.3:0/2710496066 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4f4037040 con 0x557513e3f000 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 10 client.26183 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557513ec3760 con 0x557513e3f000 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb4ef400640 1 -- 172.21.15.3:0/2710496066 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f40035f0 con 0x557513e3f000 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb4ef400640 1 -- 172.21.15.3:0/2710496066 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fb4f4032820 con 0x557513e3f000 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 10 client.26183 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 10 client.26183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 20 client.26183 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513ec3e70 0x557513ec6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 4 client.26183 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:24:40.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.195+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557513dab670 con 0x557513ec3e70 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.196+0000 7fb50e600640 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513ec3e70 0x557513ec6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.196+0000 7fb50e600640 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513ec3e70 0x557513ec6290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff7f07ab5ba8d249 server_cookie=4403944ffdef1b7d in_seq=0 out_seq=0 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.196+0000 7fb4ef400640 10 client.26183 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.196+0000 7fb4ef400640 1 -- 172.21.15.3:0/2710496066 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557513dab670 con 0x557513ec3e70 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.196+0000 7fb4ef400640 10 client.26183 handle_command_reply: tid=0 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.196+0000 7fb513086740 1 client.26183 shutdown 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3f000 msgr2=0x557513e5c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3f000 0x557513e5c3d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009780 tx=0x7fb4f40045a0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 shutdown_connections 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557513e498c0 0x557513ec1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557513e3f000 0x557513e5c3d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x557513ec3e70 0x557513ec6290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 --2- 172.21.15.3:0/2710496066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557513e3e6e0 0x557513e5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 >> 172.21.15.3:0/2710496066 conn(0x557513e3ac70 msgr2=0x557513e54860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb4ef400640 0 client.26183 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:24:40.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 shutdown_connections 2024-08-01T04:24:40.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 1 -- 172.21.15.3:0/2710496066 wait complete. 2024-08-01T04:24:40.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.197+0000 7fb513086740 20 client.26183 trim_cache size 0 max 16384 2024-08-01T04:24:40.204 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:24:40.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4f00a2d70 msgr2=0x7fb4f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4f00a2d70 0x7fb4f00a5230 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x557513e3bad0 tx=0x7fb4fc0d5530 comp rx=0 tx=0).stop 2024-08-01T04:24:40.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 msgr2=0x7fb50819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:40.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 0x7fb50819e600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00e9f0 tx=0x7fb4fc00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 shutdown_connections 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb4f00a2d70 0x7fb4f00a5230 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5081125a0 0x7fb5081a2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb508076d70 0x7fb50819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 --2- 172.21.15.3:0/1521422158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb508076450 0x7fb50819e600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 >> 172.21.15.3:0/1521422158 conn(0x7fb5080ff600 msgr2=0x7fb5081019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 shutdown_connections 2024-08-01T04:24:40.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:40.198+0000 7fb50f000640 1 -- 172.21.15.3:0/1521422158 wait complete. 2024-08-01T04:24:40.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:39 smithi163 ceph-mon[37383]: pgmap v258: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-01T04:24:41.229 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:24:41.712 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:42.081 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:41 smithi003 ceph-mon[32262]: pgmap v259: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-08-01T04:24:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:41 smithi160 ceph-mon[37224]: pgmap v259: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-08-01T04:24:42.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:41 smithi163 ceph-mon[37383]: pgmap v259: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-08-01T04:24:42.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 -- 172.21.15.3:0/3753024557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 msgr2=0x7effc0103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 --2- 172.21.15.3:0/3753024557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc0103d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effac009900 tx=0x7effac02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 -- 172.21.15.3:0/3753024557 shutdown_connections 2024-08-01T04:24:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 --2- 172.21.15.3:0/3753024557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc0104280 0x7effc01107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 --2- 172.21.15.3:0/3753024557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc0103d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 --2- 172.21.15.3:0/3753024557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc01098e0 0x7effc0109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 -- 172.21.15.3:0/3753024557 >> 172.21.15.3:0/3753024557 conn(0x7effc00ff5a0 msgr2=0x7effc01019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 -- 172.21.15.3:0/3753024557 shutdown_connections 2024-08-01T04:24:42.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.360+0000 7effc6e00640 1 -- 172.21.15.3:0/3753024557 wait complete. 2024-08-01T04:24:42.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.362+0000 7effc6e00640 1 Processor -- start 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.362+0000 7effc6e00640 1 -- start start 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.362+0000 7effc6e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.362+0000 7effc6e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 0x7effc019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc6e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc01098e0 0x7effc019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc6e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effc0113ce0 con 0x7effc0104280 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc6e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7effc0113b60 con 0x7effc01038e0 2024-08-01T04:24:42.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc6e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effc0113e60 con 0x7effc01098e0 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 0x7effc019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 0x7effc019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40916/0 (socket says 172.21.15.3:40916) 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc5000640 1 -- 172.21.15.3:0/952515414 learned_addr learned my addr 172.21.15.3:0/952515414 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc5a00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc6400640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc01098e0 0x7effc019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc5000640 1 -- 172.21.15.3:0/952515414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc01098e0 msgr2=0x7effc019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.363+0000 7effc5000640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc01098e0 0x7effc019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc5000640 1 -- 172.21.15.3:0/952515414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 msgr2=0x7effc019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc5000640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc5000640 1 -- 172.21.15.3:0/952515414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc019f1b0 con 0x7effc0104280 2024-08-01T04:24:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc5a00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc6400640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc01098e0 0x7effc019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:24:42.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc5000640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 0x7effc019a740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effac006010 tx=0x7effac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effbe000640 1 -- 172.21.15.3:0/952515414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac03f070 con 0x7effc0104280 2024-08-01T04:24:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc019f440 con 0x7effc0104280 2024-08-01T04:24:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effbe000640 1 -- 172.21.15.3:0/952515414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effac031c30 con 0x7effc0104280 2024-08-01T04:24:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.364+0000 7effbe000640 1 -- 172.21.15.3:0/952515414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac035030 con 0x7effc0104280 2024-08-01T04:24:42.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.365+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01a6d80 con 0x7effc0104280 2024-08-01T04:24:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.368+0000 7effcae0e740 1 Processor -- start 2024-08-01T04:24:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.368+0000 7effbe000640 1 -- 172.21.15.3:0/952515414 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7effac042070 con 0x7effc0104280 2024-08-01T04:24:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.368+0000 7effcae0e740 1 -- start start 2024-08-01T04:24:42.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.368+0000 7effcae0e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.368+0000 7effcae0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5b8000 0x5595be5c2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.368+0000 7effcae0e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5c28c0 0x5595be5c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effcae0e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5595be51cbf0 con 0x5595be5b76e0 2024-08-01T04:24:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effcae0e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5595be4b4b20 con 0x5595be5c28c0 2024-08-01T04:24:42.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effcae0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595be518bb0 con 0x5595be5b8000 2024-08-01T04:24:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40918/0 (socket says 172.21.15.3:40918) 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 -- 172.21.15.3:0/3911552624 learned_addr learned my addr 172.21.15.3:0/3911552624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc5a00640 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5b8000 0x5595be5c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc5000640 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5c28c0 0x5595be5c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 -- 172.21.15.3:0/3911552624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5b8000 msgr2=0x5595be5c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5b8000 0x5595be5c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 -- 172.21.15.3:0/3911552624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5c28c0 msgr2=0x5595be5c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5c28c0 0x5595be5c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc6400640 1 -- 172.21.15.3:0/3911552624 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595be4bb7f0 con 0x5595be5b76e0 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.369+0000 7effc5000640 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5c28c0 0x5595be5c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effbe000640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7effa80a2cf0 0x7effa80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effc6400640 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5b7ac0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effb8010450 tx=0x7effb8010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d38dabb2a711a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effbe000640 1 -- 172.21.15.3:0/952515414 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7effac0efc30 con 0x7effc0104280 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effc5a00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7effa80a2cf0 0x7effa80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effb2a00640 1 -- 172.21.15.3:0/3911552624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8017070 con 0x5595be5b76e0 2024-08-01T04:24:42.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effb2a00640 1 -- 172.21.15.3:0/3911552624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb8011c40 con 0x5595be5b76e0 2024-08-01T04:24:42.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.370+0000 7effb2a00640 1 -- 172.21.15.3:0/3911552624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb801adb0 con 0x5595be5b76e0 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 -- 172.21.15.3:0/3911552624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 msgr2=0x5595be5b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effc5a00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7effa80a2cf0 0x7effa80a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5595be5b4ad0 tx=0x7effb4005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5b7ac0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effb8010450 tx=0x7effb8010910 comp rx=0 tx=0).stop 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 -- 172.21.15.3:0/3911552624 shutdown_connections 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5c28c0 0x5595be5c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5b8000 0x5595be5c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 --2- 172.21.15.3:0/3911552624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5b7ac0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.371+0000 7effcae0e740 1 -- 172.21.15.3:0/3911552624 >> 172.21.15.3:0/3911552624 conn(0x5595be5b3c70 msgr2=0x5595be5b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.372+0000 7effcae0e740 1 -- 172.21.15.3:0/3911552624 shutdown_connections 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.372+0000 7effcae0e740 1 -- 172.21.15.3:0/3911552624 wait complete. 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.372+0000 7effcae0e740 1 -- start start 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.372+0000 7effcae0e740 1 Processor -- start 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5bfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5595be5c5490 con 0x5595be5b76e0 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5595be4d7cf0 con 0x5595be5b8000 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x5595be5c5610 con 0x5595be5c28c0 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effcae0e740 10 client.0 _ll_register_callbacks cb 0x5595be4c3e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effc5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effc5000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5bfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effc5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40922/0 (socket says 172.21.15.3:40922) 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.373+0000 7effc5a00640 1 -- 172.21.15.3:0/872453521 learned_addr learned my addr 172.21.15.3:0/872453521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc6400640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc5000640 1 -- 172.21.15.3:0/872453521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 msgr2=0x5595be5c0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc5000640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5c0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc5000640 1 -- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 msgr2=0x5595be5bf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc5000640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc5000640 1 -- 172.21.15.3:0/872453521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595be28e3c0 con 0x5595be5b8000 2024-08-01T04:24:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc6400640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effc5000640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5bfcc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7effac0ed2b0 tx=0x7effac0b8320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2704459b76db5cc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.374+0000 7effa7400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effa7400640 1 -- 172.21.15.3:0/872453521 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac049070 con 0x5595be5b8000 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effa7400640 1 -- 172.21.15.3:0/872453521 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7effac0b86f0 con 0x5595be5b8000 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb8011c40 con 0x5595be5b8000 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effa7400640 1 -- 172.21.15.3:0/872453521 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effac0bb9e0 con 0x5595be5b8000 2024-08-01T04:24:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effcae0e740 10 client.26195 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595be5d1110 con 0x5595be5b8000 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effa7400640 1 -- 172.21.15.3:0/872453521 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb8011c40 con 0x5595be5b8000 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effa7400640 1 -- 172.21.15.3:0/872453521 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7effac0c0080 con 0x5595be5b8000 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effcae0e740 10 client.26195 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effcae0e740 10 client.26195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.375+0000 7effcae0e740 20 client.26195 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.376+0000 7effcae0e740 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be5d2460 0x5595be5d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.376+0000 7effcae0e740 4 client.26195 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:42.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.376+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595be52a160 con 0x5595be5d2460 2024-08-01T04:24:42.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.376+0000 7effc5a00640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be5d2460 0x5595be5d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.377+0000 7effc5a00640 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be5d2460 0x5595be5d4900 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a67ad74afbb5f8f server_cookie=2a5fb64b39f3c171 in_seq=0 out_seq=0 2024-08-01T04:24:42.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.377+0000 7effa7400640 10 client.26195 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:24:42.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.377+0000 7effa7400640 1 -- 172.21.15.3:0/872453521 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595be52a160 con 0x5595be5d2460 2024-08-01T04:24:42.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.377+0000 7effa7400640 10 client.26195 handle_command_reply: tid=0 2024-08-01T04:24:42.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.377+0000 7effcae0e740 1 client.26195 shutdown 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 msgr2=0x5595be5bfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5bfcc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7effac0ed2b0 tx=0x7effac0b8320 comp rx=0 tx=0).stop 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 shutdown_connections 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be5d2460 0x5595be5d4900 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5bfcc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 --2- 172.21.15.3:0/872453521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effa7400640 0 client.26195 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:24:42.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 >> 172.21.15.3:0/872453521 conn(0x5595be5b3c70 msgr2=0x5595be5b9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:42.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 shutdown_connections 2024-08-01T04:24:42.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 1 -- 172.21.15.3:0/872453521 wait complete. 2024-08-01T04:24:42.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.378+0000 7effcae0e740 20 client.26195 trim_cache size 0 max 16384 2024-08-01T04:24:42.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 Processor -- start 2024-08-01T04:24:42.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 -- start start 2024-08-01T04:24:42.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5b9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5da220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effc01b20a0 con 0x5595be5b76e0 2024-08-01T04:24:42.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7effc01af970 con 0x5595be5b8000 2024-08-01T04:24:42.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.393+0000 7effcae0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effc01ad7e0 con 0x5595be5c28c0 2024-08-01T04:24:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc6400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5da220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:41634/0 (socket says 172.21.15.3:41634) 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 -- 172.21.15.3:0/3058702414 learned_addr learned my addr 172.21.15.3:0/3058702414 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 -- 172.21.15.3:0/3058702414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 msgr2=0x5595be5da220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5da220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 -- 172.21.15.3:0/3058702414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 msgr2=0x5595be5bf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 -- 172.21.15.3:0/3058702414 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595be5c5370 con 0x5595be5b8000 2024-08-01T04:24:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5a00640 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.394+0000 7effc5000640 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5b9840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7effac0b8a50 tx=0x7effac0f34a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d3ad5e8ae92a89 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.395+0000 7effb2000640 1 -- 172.21.15.3:0/3058702414 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac049070 con 0x5595be5b8000 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.395+0000 7effb2000640 1 -- 172.21.15.3:0/3058702414 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effac031de0 con 0x5595be5b8000 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.395+0000 7effb2000640 1 -- 172.21.15.3:0/3058702414 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0ed530 con 0x5595be5b8000 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 -- 172.21.15.3:0/3058702414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 msgr2=0x5595be5b9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5b9840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7effac0b8a50 tx=0x7effac0f34a0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 -- 172.21.15.3:0/3058702414 shutdown_connections 2024-08-01T04:24:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5da220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b8000 0x5595be5b9840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 --2- 172.21.15.3:0/3058702414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b76e0 0x5595be5bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 -- 172.21.15.3:0/3058702414 >> 172.21.15.3:0/3058702414 conn(0x5595be5b3c70 msgr2=0x5595be5b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 -- 172.21.15.3:0/3058702414 shutdown_connections 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 -- 172.21.15.3:0/3058702414 wait complete. 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 -- start start 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.396+0000 7effcae0e740 1 Processor -- start 2024-08-01T04:24:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b76e0 0x5595be5cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 0x5595be5bc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effc01ade30 con 0x5595be5b8000 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7effc01af2a0 con 0x5595be5b76e0 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7effc01ae500 con 0x5595be5c28c0 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effcae0e740 10 client.0 _ll_register_callbacks cb 0x7effac0f33f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effc5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 0x5595be5bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effc5000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 0x5595be5bc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40938/0 (socket says 172.21.15.3:40938) 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effc5000640 1 -- 172.21.15.3:0/1233539081 learned_addr learned my addr 172.21.15.3:0/1233539081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effc5a00640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b76e0 0x5595be5cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.397+0000 7effc6400640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5a00640 1 -- 172.21.15.3:0/1233539081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 msgr2=0x5595be5bc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5a00640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5bc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5a00640 1 -- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 msgr2=0x5595be5bc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5a00640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 0x5595be5bc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5a00640 1 -- 172.21.15.3:0/1233539081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595be5bcfc0 con 0x5595be5b76e0 2024-08-01T04:24:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5000640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 0x5595be5bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effc5a00640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b76e0 0x5595be5cae80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5595be5d2840 tx=0x7effb4012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac461b722c5202da server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effa7400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effa7400640 1 -- 172.21.15.3:0/1233539081 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4004020 con 0x5595be5b76e0 2024-08-01T04:24:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effa7400640 1 -- 172.21.15.3:0/1233539081 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7effb40041c0 con 0x5595be5b76e0 2024-08-01T04:24:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.398+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effac031de0 con 0x5595be5b76e0 2024-08-01T04:24:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effa7400640 1 -- 172.21.15.3:0/1233539081 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effb40043c0 con 0x5595be5b76e0 2024-08-01T04:24:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effcae0e740 10 client.26201 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595be63d110 con 0x5595be5b76e0 2024-08-01T04:24:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effa7400640 1 -- 172.21.15.3:0/1233539081 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effac031de0 con 0x5595be5b76e0 2024-08-01T04:24:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effa7400640 1 -- 172.21.15.3:0/1233539081 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7effb40047a0 con 0x5595be5b76e0 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effcae0e740 10 client.26201 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effcae0e740 10 client.26201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.399+0000 7effcae0e740 20 client.26201 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.400+0000 7effcae0e740 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be63d890 0x5595be63fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.400+0000 7effcae0e740 4 client.26201 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.400+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595be524670 con 0x5595be63d890 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.400+0000 7effc5000640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be63d890 0x5595be63fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:42.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.400+0000 7effa7400640 10 client.26201 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.400+0000 7effc5000640 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be63d890 0x5595be63fcb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9cddd5bcdcb807f server_cookie=9b0a34cf89fbdd0e in_seq=0 out_seq=0 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effa7400640 1 -- 172.21.15.3:0/1233539081 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595be524670 con 0x5595be63d890 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effa7400640 10 client.26201 handle_command_reply: tid=0 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 client.26201 shutdown 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b76e0 msgr2=0x5595be5cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b76e0 0x5595be5cae80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5595be5d2840 tx=0x7effb4012f90 comp rx=0 tx=0).stop 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 shutdown_connections 2024-08-01T04:24:42.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x5595be63d890 0x5595be63fcb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5595be5c28c0 0x5595be5bc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595be5b8000 0x5595be5bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 --2- 172.21.15.3:0/1233539081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5595be5b76e0 0x5595be5cae80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effa7400640 0 client.26201 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 >> 172.21.15.3:0/1233539081 conn(0x5595be5b3c70 msgr2=0x5595be5b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 shutdown_connections 2024-08-01T04:24:42.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 1 -- 172.21.15.3:0/1233539081 wait complete. 2024-08-01T04:24:42.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.401+0000 7effcae0e740 20 client.26201 trim_cache size 0 max 16384 2024-08-01T04:24:42.411 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:24:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7effa80a2cf0 msgr2=0x7effa80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7effa80a2cf0 0x7effa80a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5595be5b4ad0 tx=0x7effb4005f90 comp rx=0 tx=0).stop 2024-08-01T04:24:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 msgr2=0x7effc019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 0x7effc019a740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7effac006010 tx=0x7effac003c10 comp rx=0 tx=0).stop 2024-08-01T04:24:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 shutdown_connections 2024-08-01T04:24:42.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7effa80a2cf0 0x7effa80a51b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effc01098e0 0x7effc019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effc0104280 0x7effc019a740 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 --2- 172.21.15.3:0/952515414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc01038e0 0x7effc019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.402+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 >> 172.21.15.3:0/952515414 conn(0x7effc00ff5a0 msgr2=0x7effc00fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.403+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 shutdown_connections 2024-08-01T04:24:42.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:42.403+0000 7effc6e00640 1 -- 172.21.15.3:0/952515414 wait complete. 2024-08-01T04:24:43.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:24:43.787 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:44.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:43 smithi160 ceph-mon[37224]: pgmap v260: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:24:44.155 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:43 smithi003 ceph-mon[32262]: pgmap v260: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:24:44.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:43 smithi163 ceph-mon[37383]: pgmap v260: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:24:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 -- 172.21.15.3:0/693599163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5841016e0 msgr2=0x7fb584101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 --2- 172.21.15.3:0/693599163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5841016e0 0x7fb584101b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb570009890 tx=0x7fb57002a360 comp rx=0 tx=0).stop 2024-08-01T04:24:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 -- 172.21.15.3:0/693599163 shutdown_connections 2024-08-01T04:24:44.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 --2- 172.21.15.3:0/693599163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb584102080 0x7fb58410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 --2- 172.21.15.3:0/693599163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5841016e0 0x7fb584101b40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 --2- 172.21.15.3:0/693599163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 0x7fb584107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 -- 172.21.15.3:0/693599163 >> 172.21.15.3:0/693599163 conn(0x7fb5840fd3e0 msgr2=0x7fb5840ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 -- 172.21.15.3:0/693599163 shutdown_connections 2024-08-01T04:24:44.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.473+0000 7fb58a000640 1 -- 172.21.15.3:0/693599163 wait complete. 2024-08-01T04:24:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.474+0000 7fb58a000640 1 Processor -- start 2024-08-01T04:24:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 -- start start 2024-08-01T04:24:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5841016e0 0x7fb584198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 0x7fb584198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 0x7fb58419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb584111ae0 con 0x7fb5841016e0 2024-08-01T04:24:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb584111960 con 0x7fb5841076e0 2024-08-01T04:24:44.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.475+0000 7fb58a000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb584111c60 con 0x7fb584102080 2024-08-01T04:24:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 0x7fb584198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb589600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 0x7fb58419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb588c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5841016e0 0x7fb584198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb589600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 0x7fb58419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:41652/0 (socket says 172.21.15.3:41652) 2024-08-01T04:24:44.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 0x7fb584198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42468/0 (socket says 172.21.15.3:42468) 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb589600640 1 -- 172.21.15.3:0/2746774764 learned_addr learned my addr 172.21.15.3:0/2746774764 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 -- 172.21.15.3:0/2746774764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 msgr2=0x7fb58419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 0x7fb58419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 -- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5841016e0 msgr2=0x7fb584198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5841016e0 0x7fb584198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb583e00640 1 -- 172.21.15.3:0/2746774764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58419cfb0 con 0x7fb584102080 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.476+0000 7fb588c00640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5841016e0 0x7fb584198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:44.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.477+0000 7fb583e00640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 0x7fb584198540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb570035040 tx=0x7fb570033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.477+0000 7fb581600640 1 -- 172.21.15.3:0/2746774764 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570037040 con 0x7fb584102080 2024-08-01T04:24:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.477+0000 7fb581600640 1 -- 172.21.15.3:0/2746774764 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb570033d90 con 0x7fb584102080 2024-08-01T04:24:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.477+0000 7fb581600640 1 -- 172.21.15.3:0/2746774764 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570003830 con 0x7fb584102080 2024-08-01T04:24:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.477+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58419d240 con 0x7fb584102080 2024-08-01T04:24:44.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.477+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5841a4b80 con 0x7fb584102080 2024-08-01T04:24:44.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.480+0000 7fb581600640 1 -- 172.21.15.3:0/2746774764 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb570042020 con 0x7fb584102080 2024-08-01T04:24:44.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 Processor -- start 2024-08-01T04:24:44.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 -- start start 2024-08-01T04:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab2000 0x557cbaabc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaabecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557cbaa16bf0 con 0x557cbaab16e0 2024-08-01T04:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557cba9aeb20 con 0x557cbaab2000 2024-08-01T04:24:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.481+0000 7fb58e075740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557cbaa12bb0 con 0x557cbaabc8c0 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40962/0 (socket says 172.21.15.3:40962) 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 -- 172.21.15.3:0/2083093069 learned_addr learned my addr 172.21.15.3:0/2083093069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb583e00640 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaabecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb588c00640 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab2000 0x557cbaabc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 -- 172.21.15.3:0/2083093069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 msgr2=0x557cbaabecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaabecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 -- 172.21.15.3:0/2083093069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab2000 msgr2=0x557cbaabc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab2000 0x557cbaabc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb589600640 1 -- 172.21.15.3:0/2083093069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cba9b57f0 con 0x557cbaab16e0 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb588c00640 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab2000 0x557cbaabc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb581600640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb56c0a2d70 0x7fb56c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.482+0000 7fb581600640 1 -- 172.21.15.3:0/2746774764 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fb5700ee9c0 con 0x7fb584102080 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.483+0000 7fb589600640 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab1ac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e550 tx=0x7fb57c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e4c7cfba63fbf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.483+0000 7fb588c00640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb56c0a2d70 0x7fb56c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.483+0000 7fb575c00640 1 -- 172.21.15.3:0/2083093069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00d990 con 0x557cbaab16e0 2024-08-01T04:24:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.483+0000 7fb575c00640 1 -- 172.21.15.3:0/2083093069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57c01c070 con 0x557cbaab16e0 2024-08-01T04:24:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.483+0000 7fb575c00640 1 -- 172.21.15.3:0/2083093069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c016380 con 0x557cbaab16e0 2024-08-01T04:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 -- 172.21.15.3:0/2083093069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 msgr2=0x557cbaab1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab1ac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e550 tx=0x7fb57c0095c0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb588c00640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb56c0a2d70 0x7fb56c0a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb578009fd0 tx=0x7fb578005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 -- 172.21.15.3:0/2083093069 shutdown_connections 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaabecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab2000 0x557cbaabc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 --2- 172.21.15.3:0/2083093069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab1ac0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 -- 172.21.15.3:0/2083093069 >> 172.21.15.3:0/2083093069 conn(0x557cbaaadc70 msgr2=0x557cbaaae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 -- 172.21.15.3:0/2083093069 shutdown_connections 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.484+0000 7fb58e075740 1 -- 172.21.15.3:0/2083093069 wait complete. 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 -- start start 2024-08-01T04:24:44.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 Processor -- start 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 0x557cbaab9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab2000 0x557cbaab9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaaba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557cbaabf490 con 0x557cbaab2000 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x557cba9d1cf0 con 0x557cbaab16e0 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x557cbaabf610 con 0x557cbaabc8c0 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.485+0000 7fb58e075740 10 client.0 _ll_register_callbacks cb 0x557cba9bde30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb583e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab2000 0x557cbaab9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 0x557cbaab9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 0x557cbaab9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:41662/0 (socket says 172.21.15.3:41662) 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 -- 172.21.15.3:0/1681753695 learned_addr learned my addr 172.21.15.3:0/1681753695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb589600640 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaaba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 -- 172.21.15.3:0/1681753695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 msgr2=0x557cbaaba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaaba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 -- 172.21.15.3:0/1681753695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab2000 msgr2=0x557cbaab9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab2000 0x557cbaab9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.486+0000 7fb588c00640 1 -- 172.21.15.3:0/1681753695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cba7883c0 con 0x557cbaab16e0 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb588c00640 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 0x557cbaab9770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb5780042b0 tx=0x7fb578004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8464473b95af656 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb56a000640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb56a000640 1 -- 172.21.15.3:0/1681753695 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57800fb80 con 0x557cbaab16e0 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57c01c070 con 0x557cbaab16e0 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb56a000640 1 -- 172.21.15.3:0/1681753695 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fb57800fd20 con 0x557cbaab16e0 2024-08-01T04:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb56a000640 1 -- 172.21.15.3:0/1681753695 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb5780035f0 con 0x557cbaab16e0 2024-08-01T04:24:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb58e075740 10 client.26213 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.487+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cbaacf5d0 con 0x557cbaab16e0 2024-08-01T04:24:44.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb56a000640 1 -- 172.21.15.3:0/1681753695 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57c01c070 con 0x557cbaab16e0 2024-08-01T04:24:44.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb56a000640 1 -- 172.21.15.3:0/1681753695 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fb578011960 con 0x557cbaab16e0 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb58e075740 10 client.26213 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb58e075740 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb58e075740 20 client.26213 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb58e075740 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbaad09e0 0x557cbaad2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb58e075740 4 client.26213 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cbaa24160 con 0x557cbaad09e0 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.488+0000 7fb583e00640 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbaad09e0 0x557cbaad2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.489+0000 7fb583e00640 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbaad09e0 0x557cbaad2e80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ef4b9432da9bbf server_cookie=9a3504df34d430d4 in_seq=0 out_seq=0 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.489+0000 7fb56a000640 10 client.26213 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb56a000640 1 -- 172.21.15.3:0/1681753695 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557cbaa24160 con 0x557cbaad09e0 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb56a000640 10 client.26213 handle_command_reply: tid=0 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 client.26213 shutdown 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 msgr2=0x557cbaab9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 0x557cbaab9770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb5780042b0 tx=0x7fb578004730 comp rx=0 tx=0).stop 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 shutdown_connections 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbaad09e0 0x557cbaad2e80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaabc8c0 0x557cbaaba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab2000 0x557cbaab9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 --2- 172.21.15.3:0/1681753695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaab16e0 0x557cbaab9770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb56a000640 0 client.26213 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 >> 172.21.15.3:0/1681753695 conn(0x557cbaaadc70 msgr2=0x557cbaab39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 shutdown_connections 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.490+0000 7fb58e075740 1 -- 172.21.15.3:0/1681753695 wait complete. 2024-08-01T04:24:44.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.491+0000 7fb58e075740 20 client.26213 trim_cache size 0 max 16384 2024-08-01T04:24:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 Processor -- start 2024-08-01T04:24:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 -- start start 2024-08-01T04:24:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaad87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaad8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841adcc0 con 0x557cbaab16e0 2024-08-01T04:24:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841ab590 con 0x557cbaabc8c0 2024-08-01T04:24:44.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.505+0000 7fb58e075740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841a9400 con 0x557cbaab2000 2024-08-01T04:24:44.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40980/0 (socket says 172.21.15.3:40980) 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 -- 172.21.15.3:0/2210900361 learned_addr learned my addr 172.21.15.3:0/2210900361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb583e00640 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaad87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb589600640 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaad8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 -- 172.21.15.3:0/2210900361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 msgr2=0x557cbaad87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaad87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 -- 172.21.15.3:0/2210900361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 msgr2=0x557cbaad8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaad8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 -- 172.21.15.3:0/2210900361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cbaaaed20 con 0x557cbaab16e0 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.506+0000 7fb588c00640 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab4190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb578009730 tx=0x7fb57802a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85b2d2192768e30 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.507+0000 7fb575200640 1 -- 172.21.15.3:0/2210900361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578036040 con 0x557cbaab16e0 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.507+0000 7fb575200640 1 -- 172.21.15.3:0/2210900361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb578025030 con 0x557cbaab16e0 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.507+0000 7fb575200640 1 -- 172.21.15.3:0/2210900361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5780116a0 con 0x557cbaab16e0 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.507+0000 7fb58e075740 1 -- 172.21.15.3:0/2210900361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 msgr2=0x557cbaab4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.507+0000 7fb58e075740 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab4190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb578009730 tx=0x7fb57802a6b0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 -- 172.21.15.3:0/2210900361 shutdown_connections 2024-08-01T04:24:44.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaad8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaad87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 --2- 172.21.15.3:0/2210900361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaab4190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 -- 172.21.15.3:0/2210900361 >> 172.21.15.3:0/2210900361 conn(0x557cbaaadc70 msgr2=0x557cbaabb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 -- 172.21.15.3:0/2210900361 shutdown_connections 2024-08-01T04:24:44.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 -- 172.21.15.3:0/2210900361 wait complete. 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 -- start start 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.508+0000 7fb58e075740 1 Processor -- start 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaac91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaac44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaac4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841a9a50 con 0x557cbaab16e0 2024-08-01T04:24:44.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841aaec0 con 0x557cbaabc8c0 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841aa120 con 0x557cbaab2000 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb58e075740 10 client.0 _ll_register_callbacks cb 0x7fb5780040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb588c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaac91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb588c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaac91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40984/0 (socket says 172.21.15.3:40984) 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb588c00640 1 -- 172.21.15.3:0/2894172972 learned_addr learned my addr 172.21.15.3:0/2894172972 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb583e00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaac44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.509+0000 7fb589600640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaac4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb588c00640 1 -- 172.21.15.3:0/2894172972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 msgr2=0x557cbaac44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb588c00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaac44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb588c00640 1 -- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 msgr2=0x557cbaac4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb588c00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaac4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb588c00640 1 -- 172.21.15.3:0/2894172972 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cbaac4ff0 con 0x557cbaab16e0 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb583e00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaac44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb588c00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaac91c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb57802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9816bc61992e8d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb56a000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:24:44.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.510+0000 7fb56a000640 1 -- 172.21.15.3:0/2894172972 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578036040 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb56a000640 1 -- 172.21.15.3:0/2894172972 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fb578003790 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578025030 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb56a000640 1 -- 172.21.15.3:0/2894172972 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb578003990 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cbab3b4e0 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb56a000640 1 -- 172.21.15.3:0/2894172972 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb578025030 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb56a000640 1 -- 172.21.15.3:0/2894172972 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fb578042030 con 0x557cbaab16e0 2024-08-01T04:24:44.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 20 client.16032 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbab3bbe0 0x557cbab3e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 4 client.16032 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.511+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cbaa1e670 con 0x557cbab3bbe0 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.512+0000 7fb583e00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbab3bbe0 0x557cbab3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.512+0000 7fb583e00640 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbab3bbe0 0x557cbab3e000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8e841f98d0083a server_cookie=2903b41ec03e59af in_seq=0 out_seq=0 2024-08-01T04:24:44.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.512+0000 7fb56a000640 10 client.16032 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb56a000640 1 -- 172.21.15.3:0/2894172972 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cbaa1e670 con 0x557cbab3bbe0 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb56a000640 10 client.16032 handle_command_reply: tid=0 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 client.16032 shutdown 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 msgr2=0x557cbaac91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaac91c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb57802a420 comp rx=0 tx=0).stop 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 shutdown_connections 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x557cbab3bbe0 0x557cbab3e000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557cbaabc8c0 0x557cbaac4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557cbaab2000 0x557cbaac44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 --2- 172.21.15.3:0/2894172972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557cbaab16e0 0x557cbaac91c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb56a000640 0 client.16032 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 >> 172.21.15.3:0/2894172972 conn(0x557cbaaadc70 msgr2=0x557cbaabafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 shutdown_connections 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 1 -- 172.21.15.3:0/2894172972 wait complete. 2024-08-01T04:24:44.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.513+0000 7fb58e075740 20 client.16032 trim_cache size 0 max 16384 2024-08-01T04:24:44.523 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:24:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb56c0a2d70 msgr2=0x7fb56c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb56c0a2d70 0x7fb56c0a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb578009fd0 tx=0x7fb578005f90 comp rx=0 tx=0).stop 2024-08-01T04:24:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 msgr2=0x7fb584198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:44.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 0x7fb584198540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb570035040 tx=0x7fb570033ae0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 shutdown_connections 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5841076e0 0x7fb58419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fb56c0a2d70 0x7fb56c0a5230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb584102080 0x7fb584198540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 --2- 172.21.15.3:0/2746774764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5841016e0 0x7fb584198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 >> 172.21.15.3:0/2746774764 conn(0x7fb5840fd3e0 msgr2=0x7fb5840fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 shutdown_connections 2024-08-01T04:24:44.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:44.514+0000 7fb58a000640 1 -- 172.21.15.3:0/2746774764 wait complete. 2024-08-01T04:24:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:45 smithi160 ceph-mon[37224]: pgmap v261: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:24:46.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:45 smithi163 ceph-mon[37383]: pgmap v261: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:24:46.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:45 smithi003 ceph-mon[32262]: pgmap v261: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:24:46.396 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:24:46.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:24:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 -- 172.21.15.3:0/417886090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb40104280 msgr2=0x7feb401107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 --2- 172.21.15.3:0/417886090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb40104280 0x7feb401107b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feb38009cd0 tx=0x7feb3802a8f0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 -- 172.21.15.3:0/417886090 shutdown_connections 2024-08-01T04:24:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 --2- 172.21.15.3:0/417886090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb40104280 0x7feb401107b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 --2- 172.21.15.3:0/417886090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 --2- 172.21.15.3:0/417886090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb401098e0 0x7feb40109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 -- 172.21.15.3:0/417886090 >> 172.21.15.3:0/417886090 conn(0x7feb400ff5a0 msgr2=0x7feb401019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 -- 172.21.15.3:0/417886090 shutdown_connections 2024-08-01T04:24:47.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.611+0000 7feb46000640 1 -- 172.21.15.3:0/417886090 wait complete. 2024-08-01T04:24:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 Processor -- start 2024-08-01T04:24:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 -- start start 2024-08-01T04:24:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40076430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb40104280 0x7feb40076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb401098e0 0x7feb40076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb40113ce0 con 0x7feb401098e0 2024-08-01T04:24:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feb40113b60 con 0x7feb401038e0 2024-08-01T04:24:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.612+0000 7feb46000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feb40113e60 con 0x7feb40104280 2024-08-01T04:24:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40076430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb45600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb401098e0 0x7feb40076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40076430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:41710/0 (socket says 172.21.15.3:41710) 2024-08-01T04:24:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 -- 172.21.15.3:0/1954932980 learned_addr learned my addr 172.21.15.3:0/1954932980 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb37e00640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb40104280 0x7feb40076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 -- 172.21.15.3:0/1954932980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb40104280 msgr2=0x7feb40076970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb40104280 0x7feb40076970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 -- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb401098e0 msgr2=0x7feb40076eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb401098e0 0x7feb40076eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.613+0000 7feb44c00640 1 -- 172.21.15.3:0/1954932980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb401aa820 con 0x7feb401038e0 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.614+0000 7feb44c00640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40076430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feb2400ea50 tx=0x7feb2400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.614+0000 7feb3e000640 1 -- 172.21.15.3:0/1954932980 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2400cd60 con 0x7feb401038e0 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.614+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb401aab10 con 0x7feb401038e0 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.614+0000 7feb3e000640 1 -- 172.21.15.3:0/1954932980 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb24010910 con 0x7feb401038e0 2024-08-01T04:24:47.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.614+0000 7feb3e000640 1 -- 172.21.15.3:0/1954932980 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24018950 con 0x7feb401038e0 2024-08-01T04:24:47.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.614+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401ab050 con 0x7feb401038e0 2024-08-01T04:24:47.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 Processor -- start 2024-08-01T04:24:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 -- start start 2024-08-01T04:24:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aaf70 0x562e599b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562e598c57e0 con 0x562e599b5830 2024-08-01T04:24:47.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562e59910ac0 con 0x562e599aaf70 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.618+0000 7feb4a09b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562e598a88c0 con 0x562e599aa650 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb3e000640 1 -- 172.21.15.3:0/1954932980 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7feb24010ac0 con 0x7feb401038e0 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42524/0 (socket says 172.21.15.3:42524) 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 -- 172.21.15.3:0/1321176000 learned_addr learned my addr 172.21.15.3:0/1321176000 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb37e00640 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb44c00640 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aaf70 0x562e599b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 -- 172.21.15.3:0/1321176000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aaf70 msgr2=0x562e599b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aaf70 0x562e599b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 -- 172.21.15.3:0/1321176000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 msgr2=0x562e599b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.619+0000 7feb45600640 1 -- 172.21.15.3:0/1321176000 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e598af760 con 0x562e599aa650 2024-08-01T04:24:47.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.620+0000 7feb45600640 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599aaa30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb38004aa0 tx=0x7feb38009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47fcef1abd78fd99 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:47.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.620+0000 7feb35c00640 1 -- 172.21.15.3:0/1321176000 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38030660 con 0x562e599aa650 2024-08-01T04:24:47.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.620+0000 7feb35c00640 1 -- 172.21.15.3:0/1321176000 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb38036030 con 0x562e599aa650 2024-08-01T04:24:47.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.620+0000 7feb35c00640 1 -- 172.21.15.3:0/1321176000 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380043a0 con 0x562e599aa650 2024-08-01T04:24:47.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 -- 172.21.15.3:0/1321176000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 msgr2=0x562e599aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599aaa30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb38004aa0 tx=0x7feb38009360 comp rx=0 tx=0).stop 2024-08-01T04:24:47.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 -- 172.21.15.3:0/1321176000 shutdown_connections 2024-08-01T04:24:47.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aaf70 0x562e599b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1321176000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599aaa30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 -- 172.21.15.3:0/1321176000 >> 172.21.15.3:0/1321176000 conn(0x562e599a6be0 msgr2=0x562e599a6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 -- 172.21.15.3:0/1321176000 shutdown_connections 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb4a09b740 1 -- 172.21.15.3:0/1321176000 wait complete. 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.621+0000 7feb3e000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.622+0000 7feb3e000640 1 -- 172.21.15.3:0/1954932980 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7feb240cf780 con 0x7feb401038e0 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.622+0000 7feb37e00640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.622+0000 7feb4a09b740 1 -- start start 2024-08-01T04:24:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.622+0000 7feb4a09b740 1 Processor -- start 2024-08-01T04:24:47.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb37e00640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x562e599a7f60 tx=0x7feb2c005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599b5830 0x562e599b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562e598cbae0 con 0x562e599aaf70 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x562e5990ca80 con 0x562e599aa650 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x562e599b84a0 con 0x562e599b5830 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb4a09b740 10 client.0 _ll_register_callbacks cb 0x562e598cd190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:47.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41016/0 (socket says 172.21.15.3:41016) 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb37e00640 1 -- 172.21.15.3:0/1706863969 learned_addr learned my addr 172.21.15.3:0/1706863969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.623+0000 7feb44c00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb45600640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599b5830 0x562e599b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb44c00640 1 -- 172.21.15.3:0/1706863969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599b5830 msgr2=0x562e599b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb44c00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599b5830 0x562e599b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb44c00640 1 -- 172.21.15.3:0/1706863969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 msgr2=0x562e599b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb44c00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb44c00640 1 -- 172.21.15.3:0/1706863969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e596826a0 con 0x562e599aa650 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb37e00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb45600640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599b5830 0x562e599b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb44c00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599b2650 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb240cce00 tx=0x7feb2409b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29dbf53fa8151117 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb32a00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb32a00640 1 -- 172.21.15.3:0/1706863969 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240cd290 con 0x562e599aa650 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb32a00640 1 -- 172.21.15.3:0/1706863969 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7feb240cfd70 con 0x562e599aa650 2024-08-01T04:24:47.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.624+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb38036030 con 0x562e599aa650 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb32a00640 1 -- 172.21.15.3:0/1706863969 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb2409ff80 con 0x562e599aa650 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 10 client.26231 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562e599c3f00 con 0x562e599aa650 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb32a00640 1 -- 172.21.15.3:0/1706863969 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb38036030 con 0x562e599aa650 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb32a00640 1 -- 172.21.15.3:0/1706863969 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7feb24018c80 con 0x562e599aa650 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 10 client.26231 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:47.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 20 client.26231 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e599c53c0 0x562e599c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 4 client.26231 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.625+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e5991e030 con 0x562e599c53c0 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.626+0000 7feb37e00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e599c53c0 0x562e599c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.626+0000 7feb37e00640 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e599c53c0 0x562e599c7810 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5d99f22ed8b70b server_cookie=32aafa78239885f5 in_seq=0 out_seq=0 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.626+0000 7feb32a00640 10 client.26231 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:47.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.627+0000 7feb32a00640 1 -- 172.21.15.3:0/1706863969 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e5991e030 con 0x562e599c53c0 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.627+0000 7feb32a00640 10 client.26231 handle_command_reply: tid=0 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.627+0000 7feb4a09b740 1 client.26231 shutdown 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 msgr2=0x562e599b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599b2650 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb240cce00 tx=0x7feb2409b880 comp rx=0 tx=0).stop 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 shutdown_connections 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e599c53c0 0x562e599c7810 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599b5830 0x562e599b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1706863969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599b2650 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb32a00640 0 client.26231 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 >> 172.21.15.3:0/1706863969 conn(0x562e599a6be0 msgr2=0x562e599ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 shutdown_connections 2024-08-01T04:24:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 1 -- 172.21.15.3:0/1706863969 wait complete. 2024-08-01T04:24:47.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.628+0000 7feb4a09b740 20 client.26231 trim_cache size 0 max 16384 2024-08-01T04:24:47.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.647+0000 7feb4a09b740 1 Processor -- start 2024-08-01T04:24:47.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.647+0000 7feb4a09b740 1 -- start start 2024-08-01T04:24:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 0x562e599cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb4a09b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b64d0 con 0x562e599aaf70 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb4a09b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b3da0 con 0x562e599b5830 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb4a09b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b1c10 con 0x562e599aa650 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb45600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 0x562e599cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb37e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb45600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 0x562e599cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:41726/0 (socket says 172.21.15.3:41726) 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb45600640 1 -- 172.21.15.3:0/1515492691 learned_addr learned my addr 172.21.15.3:0/1515492691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:47.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.648+0000 7feb44c00640 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb45600640 1 -- 172.21.15.3:0/1515492691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 msgr2=0x562e599ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb45600640 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb45600640 1 -- 172.21.15.3:0/1515492691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 msgr2=0x562e599cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb45600640 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb45600640 1 -- 172.21.15.3:0/1515492691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e599aada0 con 0x562e599b5830 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb45600640 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 0x562e599cd620 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb38009430 tx=0x7feb38000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a74273a15d0b19 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb35200640 1 -- 172.21.15.3:0/1515492691 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38030460 con 0x562e599b5830 2024-08-01T04:24:47.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb35200640 1 -- 172.21.15.3:0/1515492691 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb38036040 con 0x562e599b5830 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.649+0000 7feb35200640 1 -- 172.21.15.3:0/1515492691 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380042a0 con 0x562e599b5830 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 -- 172.21.15.3:0/1515492691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 msgr2=0x562e599cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 0x562e599cd620 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb38009430 tx=0x7feb38000f40 comp rx=0 tx=0).stop 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 -- 172.21.15.3:0/1515492691 shutdown_connections 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599b5830 0x562e599cd620 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599aaf70 0x562e599cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1515492691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aa650 0x562e599ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 -- 172.21.15.3:0/1515492691 >> 172.21.15.3:0/1515492691 conn(0x562e599a6be0 msgr2=0x562e599b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 -- 172.21.15.3:0/1515492691 shutdown_connections 2024-08-01T04:24:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.650+0000 7feb4a09b740 1 -- 172.21.15.3:0/1515492691 wait complete. 2024-08-01T04:24:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.651+0000 7feb4a09b740 1 -- start start 2024-08-01T04:24:47.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.651+0000 7feb4a09b740 1 Processor -- start 2024-08-01T04:24:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.651+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aaf70 0x562e599af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb4a09b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb4a09b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b2260 con 0x562e599b5830 2024-08-01T04:24:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb4a09b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b36d0 con 0x562e599aa650 2024-08-01T04:24:47.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb4a09b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feb401b2930 con 0x562e599aaf70 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb4a09b740 10 client.0 _ll_register_callbacks cb 0x7feb2409ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb45600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb44c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb44c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599bdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:41738/0 (socket says 172.21.15.3:41738) 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb44c00640 1 -- 172.21.15.3:0/1097835974 learned_addr learned my addr 172.21.15.3:0/1097835974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb37e00640 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aaf70 0x562e599af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb45600640 1 -- 172.21.15.3:0/1097835974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aaf70 msgr2=0x562e599af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb45600640 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aaf70 0x562e599af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb45600640 1 -- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 msgr2=0x562e599bdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb45600640 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599bdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.652+0000 7feb45600640 1 -- 172.21.15.3:0/1097835974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e59a2f0b0 con 0x562e599b5830 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb45600640 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599afd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb3802ae10 tx=0x7feb38003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425b99330de0ac68 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb32a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb32a00640 1 -- 172.21.15.3:0/1097835974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38030b40 con 0x562e599b5830 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb32a00640 1 -- 172.21.15.3:0/1097835974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7feb38030ce0 con 0x562e599b5830 2024-08-01T04:24:47.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb38036040 con 0x562e599b5830 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb32a00640 1 -- 172.21.15.3:0/1097835974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb38045440 con 0x562e599b5830 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb4a09b740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.653+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562e59a302b0 con 0x562e599b5830 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb32a00640 1 -- 172.21.15.3:0/1097835974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb38036040 con 0x562e599b5830 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb32a00640 1 -- 172.21.15.3:0/1097835974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7feb38043070 con 0x562e599b5830 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb4a09b740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb4a09b740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb4a09b740 20 client.16050 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:47.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e59a30530 0x562e59a32920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb4a09b740 4 client.16050 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:24:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e59918540 con 0x562e59a30530 2024-08-01T04:24:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.654+0000 7feb44c00640 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e59a30530 0x562e59a32920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.655+0000 7feb44c00640 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e59a30530 0x562e59a32920 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea40d729e4dc7290 server_cookie=e5ff8d42c12b9ac7 in_seq=0 out_seq=0 2024-08-01T04:24:47.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.655+0000 7feb32a00640 10 client.16050 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:47.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.656+0000 7feb32a00640 1 -- 172.21.15.3:0/1097835974 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e59918540 con 0x562e59a30530 2024-08-01T04:24:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.656+0000 7feb32a00640 10 client.16050 handle_command_reply: tid=0 2024-08-01T04:24:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.656+0000 7feb4a09b740 1 client.16050 shutdown 2024-08-01T04:24:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 msgr2=0x562e599afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599afd20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb3802ae10 tx=0x7feb38003ba0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 shutdown_connections 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x562e59a30530 0x562e59a32920 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562e599b5830 0x562e599afd20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562e599aaf70 0x562e599af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 --2- 172.21.15.3:0/1097835974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x562e599aa650 0x562e599bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 >> 172.21.15.3:0/1097835974 conn(0x562e599a6be0 msgr2=0x562e599a5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb32a00640 0 client.16050 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:47.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 shutdown_connections 2024-08-01T04:24:47.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 1 -- 172.21.15.3:0/1097835974 wait complete. 2024-08-01T04:24:47.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.657+0000 7feb4a09b740 20 client.16050 trim_cache size 0 max 16384 2024-08-01T04:24:47.664 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:24:47.664 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:24:47.664 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "c137a699-8ffd-44e3-b84e-ffc8358c9c76", 2024-08-01T04:24:47.664 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:24:47.665 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:24:47.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feb1c0a2c00 msgr2=0x7feb1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x562e599a7f60 tx=0x7feb2c005f90 comp rx=0 tx=0).stop 2024-08-01T04:24:47.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 msgr2=0x7feb40076430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40076430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feb2400ea50 tx=0x7feb2400ef10 comp rx=0 tx=0).stop 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 shutdown_connections 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb401098e0 0x7feb40076eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb40104280 0x7feb40076970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 --2- 172.21.15.3:0/1954932980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb401038e0 0x7feb40076430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:47.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.658+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 >> 172.21.15.3:0/1954932980 conn(0x7feb400ff5a0 msgr2=0x7feb400ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:47.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.659+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 shutdown_connections 2024-08-01T04:24:47.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:47.659+0000 7feb46000640 1 -- 172.21.15.3:0/1954932980 wait complete. 2024-08-01T04:24:48.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:47 smithi163 ceph-mon[37383]: pgmap v262: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:24:48.110 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:47 smithi003 ceph-mon[32262]: pgmap v262: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:24:48.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:47 smithi160 ceph-mon[37224]: pgmap v262: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:24:48.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:24:48.993 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:24:49.151 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:24:49.151 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.151 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:24:49.151 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:48 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:24:49.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:24:49.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:24:49.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:48 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:24:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:24:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:24:49.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:48 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:24:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.659+0000 7f2d76400640 1 -- 172.21.15.3:0/3981538345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 msgr2=0x7f2d701107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.659+0000 7f2d76400640 1 --2- 172.21.15.3:0/3981538345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 0x7f2d701107b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009cd0 tx=0x7f2d6802a8f0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 -- 172.21.15.3:0/3981538345 shutdown_connections 2024-08-01T04:24:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 --2- 172.21.15.3:0/3981538345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 0x7f2d701107b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 --2- 172.21.15.3:0/3981538345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d701038e0 0x7f2d70103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 --2- 172.21.15.3:0/3981538345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701098e0 0x7f2d70109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 -- 172.21.15.3:0/3981538345 >> 172.21.15.3:0/3981538345 conn(0x7f2d700ff5a0 msgr2=0x7f2d701019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 -- 172.21.15.3:0/3981538345 shutdown_connections 2024-08-01T04:24:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.660+0000 7f2d76400640 1 -- 172.21.15.3:0/3981538345 wait complete. 2024-08-01T04:24:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.661+0000 7f2d76400640 1 Processor -- start 2024-08-01T04:24:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.661+0000 7f2d76400640 1 -- start start 2024-08-01T04:24:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d76400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 0x7f2d7019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d76400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 0x7f2d7019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d76400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d701098e0 0x7f2d7019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d76400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70113ce0 con 0x7f2d70104280 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d76400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70113b60 con 0x7f2d701098e0 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d76400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70113e60 con 0x7f2d701038e0 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d75000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 0x7f2d7019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 0x7f2d7019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42554/0 (socket says 172.21.15.3:42554) 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 -- 172.21.15.3:0/1538386663 learned_addr learned my addr 172.21.15.3:0/1538386663 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.662+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 0x7f2d7019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75a00640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d701098e0 0x7f2d7019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 -- 172.21.15.3:0/1538386663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d701098e0 msgr2=0x7f2d7019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d701098e0 0x7f2d7019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 -- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 msgr2=0x7f2d7019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 0x7f2d7019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 -- 172.21.15.3:0/1538386663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7019f130 con 0x7f2d701038e0 2024-08-01T04:24:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.663+0000 7f2d75000640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 0x7f2d7019a180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e9f0 tx=0x7f2d6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.664+0000 7f2d6d600640 1 -- 172.21.15.3:0/1538386663 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400cd80 con 0x7f2d701038e0 2024-08-01T04:24:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.664+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7019f420 con 0x7f2d701038e0 2024-08-01T04:24:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.664+0000 7f2d6d600640 1 -- 172.21.15.3:0/1538386663 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d64010910 con 0x7f2d701038e0 2024-08-01T04:24:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.664+0000 7f2d6d600640 1 -- 172.21.15.3:0/1538386663 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640188f0 con 0x7f2d701038e0 2024-08-01T04:24:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.664+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a6d00 con 0x7f2d701038e0 2024-08-01T04:24:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.668+0000 7f2d6d600640 1 -- 172.21.15.3:0/1538386663 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2d64010ac0 con 0x7f2d701038e0 2024-08-01T04:24:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.668+0000 7f2d7a5c9740 1 Processor -- start 2024-08-01T04:24:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.668+0000 7f2d7a5c9740 1 -- start start 2024-08-01T04:24:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.668+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d78160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.668+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.668+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d82f60 0x55b011d85350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b011cddbf0 con 0x55b011d82f60 2024-08-01T04:24:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b011c75bd0 con 0x55b011d786a0 2024-08-01T04:24:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b011cd9bb0 con 0x55b011d77d80 2024-08-01T04:24:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d78160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d82f60 0x55b011d85350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d78160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42560/0 (socket says 172.21.15.3:42560) 2024-08-01T04:24:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 -- 172.21.15.3:0/3793163390 learned_addr learned my addr 172.21.15.3:0/3793163390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 -- 172.21.15.3:0/3793163390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 msgr2=0x55b011d82a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d82a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 -- 172.21.15.3:0/3793163390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d82f60 msgr2=0x55b011d85350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d82f60 0x55b011d85350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d75a00640 1 -- 172.21.15.3:0/3793163390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b011c7c8a0 con 0x55b011d77d80 2024-08-01T04:24:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.669+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d82f60 0x55b011d85350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d78160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d68035000 tx=0x7f2d68009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e7348ecebd7934 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d6d600640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2d580a2d70 0x7f2d580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d6d600640 1 -- 172.21.15.3:0/1538386663 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f2d640cf850 con 0x7f2d701038e0 2024-08-01T04:24:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d5e000640 1 -- 172.21.15.3:0/3793163390 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680305a0 con 0x55b011d77d80 2024-08-01T04:24:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d75000640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2d580a2d70 0x7f2d580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d5e000640 1 -- 172.21.15.3:0/3793163390 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d68036040 con 0x55b011d77d80 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.670+0000 7f2d5e000640 1 -- 172.21.15.3:0/3793163390 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68004350 con 0x55b011d77d80 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3793163390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 msgr2=0x55b011d78160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d78160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d68035000 tx=0x7f2d68009a20 comp rx=0 tx=0).stop 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d75000640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2d580a2d70 0x7f2d580a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55b011d75170 tx=0x7f2d640d5530 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3793163390 shutdown_connections 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d82f60 0x55b011d85350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3793163390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d78160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3793163390 >> 172.21.15.3:0/3793163390 conn(0x55b011d74310 msgr2=0x55b011d746f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3793163390 shutdown_connections 2024-08-01T04:24:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.671+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3793163390 wait complete. 2024-08-01T04:24:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.672+0000 7f2d7a5c9740 1 -- start start 2024-08-01T04:24:49.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.672+0000 7f2d7a5c9740 1 Processor -- start 2024-08-01T04:24:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d82f60 0x55b011d808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b011d85b30 con 0x55b011d786a0 2024-08-01T04:24:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b011c98d40 con 0x55b011d82f60 2024-08-01T04:24:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b011d85cb0 con 0x55b011d77d80 2024-08-01T04:24:49.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d7a5c9740 10 client.0 _ll_register_callbacks cb 0x55b011c61b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d75a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d75a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d7fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:42572/0 (socket says 172.21.15.3:42572) 2024-08-01T04:24:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d75a00640 1 -- 172.21.15.3:0/322374281 learned_addr learned my addr 172.21.15.3:0/322374281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.673+0000 7f2d75a00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d82f60 0x55b011d808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d75a00640 1 -- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d82f60 msgr2=0x55b011d808b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d75a00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d82f60 0x55b011d808b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d75a00640 1 -- 172.21.15.3:0/322374281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 msgr2=0x55b011d80370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d75a00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d75a00640 1 -- 172.21.15.3:0/322374281 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b011a4f3c0 con 0x55b011d77d80 2024-08-01T04:24:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:24:49.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d75a00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d7fe30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009620 tx=0x7f2d6803e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d4992c9dd639526 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d52a00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:24:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d52a00640 1 -- 172.21.15.3:0/322374281 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68059060 con 0x55b011d77d80 2024-08-01T04:24:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d52a00640 1 -- 172.21.15.3:0/322374281 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f2d680305a0 con 0x55b011d77d80 2024-08-01T04:24:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.674+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d68036040 con 0x55b011d77d80 2024-08-01T04:24:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d52a00640 1 -- 172.21.15.3:0/322374281 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d68004b40 con 0x55b011d77d80 2024-08-01T04:24:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 10 client.26224 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b011d85b30 con 0x55b011d77d80 2024-08-01T04:24:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d52a00640 1 -- 172.21.15.3:0/322374281 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d68036040 con 0x55b011d77d80 2024-08-01T04:24:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d52a00640 1 -- 172.21.15.3:0/322374281 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2d68004690 con 0x55b011d77d80 2024-08-01T04:24:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 10 client.26224 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:49.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 20 client.26224 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011d92940 0x55b011d94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 4 client.26224 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:24:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.675+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b011ceb160 con 0x55b011d92940 2024-08-01T04:24:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.676+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011d92940 0x55b011d94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.676+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011d92940 0x55b011d94d30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c1eb30a369d347 server_cookie=94ccd5ebd4affe65 in_seq=0 out_seq=0 2024-08-01T04:24:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.676+0000 7f2d52a00640 10 client.26224 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d52a00640 1 -- 172.21.15.3:0/322374281 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b011ceb160 con 0x55b011d92940 2024-08-01T04:24:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d52a00640 10 client.26224 handle_command_reply: tid=0 2024-08-01T04:24:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 client.26224 shutdown 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 msgr2=0x55b011d7fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d7fe30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009620 tx=0x7f2d6803e040 comp rx=0 tx=0).stop 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 shutdown_connections 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011d92940 0x55b011d94d30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d82f60 0x55b011d808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/322374281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d77d80 0x55b011d7fe30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d52a00640 0 client.26224 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 >> 172.21.15.3:0/322374281 conn(0x55b011d74310 msgr2=0x55b011d79ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 shutdown_connections 2024-08-01T04:24:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/322374281 wait complete. 2024-08-01T04:24:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.678+0000 7f2d7a5c9740 20 client.26224 trim_cache size 0 max 16384 2024-08-01T04:24:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 Processor -- start 2024-08-01T04:24:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 -- start start 2024-08-01T04:24:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d77d80 0x55b011d7a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d9a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701b2210 con 0x55b011d786a0 2024-08-01T04:24:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701afae0 con 0x55b011d77d80 2024-08-01T04:24:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701ad950 con 0x55b011d82f60 2024-08-01T04:24:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d6fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41076/0 (socket says 172.21.15.3:41076) 2024-08-01T04:24:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2223869382 learned_addr learned my addr 172.21.15.3:0/2223869382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d75a00640 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d77d80 0x55b011d7a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.698+0000 7f2d75a00640 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d9a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2223869382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 msgr2=0x55b011d9a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d9a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2223869382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d77d80 msgr2=0x55b011d7a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d77d80 0x55b011d7a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d6fe00640 1 -- 172.21.15.3:0/2223869382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b011d7e3a0 con 0x55b011d786a0 2024-08-01T04:24:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55b011d75690 tx=0x7f2d60005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96f7e27a7cbbaf7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.699+0000 7f2d5d600640 1 -- 172.21.15.3:0/2223869382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60005020 con 0x55b011d786a0 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d5d600640 1 -- 172.21.15.3:0/2223869382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6000f7f0 con 0x55b011d786a0 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d5d600640 1 -- 172.21.15.3:0/2223869382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004a00 con 0x55b011d786a0 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/2223869382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 msgr2=0x55b011d80700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55b011d75690 tx=0x7f2d60005f90 comp rx=0 tx=0).stop 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/2223869382 shutdown_connections 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d9a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d786a0 0x55b011d80700 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/2223869382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d77d80 0x55b011d7a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/2223869382 >> 172.21.15.3:0/2223869382 conn(0x55b011d74310 msgr2=0x55b011d81eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.700+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/2223869382 shutdown_connections 2024-08-01T04:24:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.701+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/2223869382 wait complete. 2024-08-01T04:24:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.701+0000 7f2d7a5c9740 1 -- start start 2024-08-01T04:24:49.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.701+0000 7f2d7a5c9740 1 Processor -- start 2024-08-01T04:24:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 0x55b011d8b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701adfa0 con 0x55b011d77d80 2024-08-01T04:24:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701af410 con 0x55b011d786a0 2024-08-01T04:24:49.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701ae670 con 0x55b011d82f60 2024-08-01T04:24:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d7a5c9740 10 client.0 _ll_register_callbacks cb 0x7f2d600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:24:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d75a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 0x55b011d8b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d75a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 0x55b011d8b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41080/0 (socket says 172.21.15.3:41080) 2024-08-01T04:24:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d75a00640 1 -- 172.21.15.3:0/3519647239 learned_addr learned my addr 172.21.15.3:0/3519647239 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:24:49.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.702+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d75a00640 1 -- 172.21.15.3:0/3519647239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 msgr2=0x55b011d7cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d7cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d75a00640 1 -- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 msgr2=0x55b011d7ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d7ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d75a00640 1 -- 172.21.15.3:0/3519647239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b011d7d5f0 con 0x55b011d77d80 2024-08-01T04:24:49.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d75a00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 0x55b011d8b4d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009360 tx=0x7f2d68003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f7dd9bc60ef562 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:24:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d52a00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:24:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d52a00640 1 -- 172.21.15.3:0/3519647239 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68062040 con 0x55b011d77d80 2024-08-01T04:24:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6000f7f0 con 0x55b011d77d80 2024-08-01T04:24:49.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.703+0000 7f2d52a00640 1 -- 172.21.15.3:0/3519647239 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f2d680305a0 con 0x55b011d77d80 2024-08-01T04:24:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d52a00640 1 -- 172.21.15.3:0/3519647239 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d68059040 con 0x55b011d77d80 2024-08-01T04:24:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 10 client.16068 fetch_fsmap learned FSMap version 29 2024-08-01T04:24:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b011dfd230 con 0x55b011d77d80 2024-08-01T04:24:49.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d52a00640 1 -- 172.21.15.3:0/3519647239 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6000f7f0 con 0x55b011d77d80 2024-08-01T04:24:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d52a00640 1 -- 172.21.15.3:0/3519647239 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2d680309c0 con 0x55b011d77d80 2024-08-01T04:24:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 10 client.16068 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:24:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:24:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 20 client.16068 populate_metadata read hostname 'smithi003' 2024-08-01T04:24:49.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011dfd8f0 0x55b011dffce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:24:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 4 client.16068 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:24:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.704+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b011ce5670 con 0x55b011dfd8f0 2024-08-01T04:24:49.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.705+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011dfd8f0 0x55b011dffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:24:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.705+0000 7f2d6fe00640 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011dfd8f0 0x55b011dffce0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2e5c23b28ed815 server_cookie=b001e83bda669713 in_seq=0 out_seq=0 2024-08-01T04:24:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.705+0000 7f2d52a00640 10 client.16068 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:49.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.705+0000 7f2d52a00640 1 -- 172.21.15.3:0/3519647239 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b011ce5670 con 0x55b011dfd8f0 2024-08-01T04:24:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.705+0000 7f2d52a00640 10 client.16068 handle_command_reply: tid=0 2024-08-01T04:24:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.705+0000 7f2d7a5c9740 1 client.16068 shutdown 2024-08-01T04:24:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 msgr2=0x55b011d8b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 0x55b011d8b4d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009360 tx=0x7f2d68003eb0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 shutdown_connections 2024-08-01T04:24:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b011dfd8f0 0x55b011dffce0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b011d82f60 0x55b011d7cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b011d786a0 0x55b011d7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 --2- 172.21.15.3:0/3519647239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b011d77d80 0x55b011d8b4d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d52a00640 0 client.16068 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:24:49.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 >> 172.21.15.3:0/3519647239 conn(0x55b011d74310 msgr2=0x55b011d72bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:49.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 shutdown_connections 2024-08-01T04:24:49.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 1 -- 172.21.15.3:0/3519647239 wait complete. 2024-08-01T04:24:49.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.706+0000 7f2d7a5c9740 20 client.16068 trim_cache size 0 max 16384 2024-08-01T04:24:49.720 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:24:49.722 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:24:49.722 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:24:49.723 INFO:teuthology.orchestra.run.smithi003.stdout: "c137a699-8ffd-44e3-b84e-ffc8358c9c76": { 2024-08-01T04:24:49.723 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:24:49.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "c137a699-8ffd-44e3-b84e-ffc8358c9c76", 2024-08-01T04:24:49.723 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:24:49.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:24:49.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:24:49.724 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:24:49.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2d580a2d70 msgr2=0x7f2d580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2d580a2d70 0x7f2d580a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55b011d75170 tx=0x7f2d640d5530 comp rx=0 tx=0).stop 2024-08-01T04:24:49.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 msgr2=0x7f2d7019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:24:49.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 0x7f2d7019a180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400e9f0 tx=0x7f2d6400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 shutdown_connections 2024-08-01T04:24:49.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d701098e0 0x7f2d7019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d70104280 0x7f2d7019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f2d580a2d70 0x7f2d580a5230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 --2- 172.21.15.3:0/1538386663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d701038e0 0x7f2d7019a180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:24:49.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 >> 172.21.15.3:0/1538386663 conn(0x7f2d700ff5a0 msgr2=0x7f2d700ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:24:49.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 shutdown_connections 2024-08-01T04:24:49.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:24:49.707+0000 7f2d76400640 1 -- 172.21.15.3:0/1538386663 wait complete. 2024-08-01T04:24:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:49 smithi003 ceph-mon[32262]: pgmap v263: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:24:50.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:49 smithi160 ceph-mon[37224]: pgmap v263: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:24:50.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:49 smithi163 ceph-mon[37383]: pgmap v263: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:24:50.553 INFO:tasks.cephfs.filesystem:scrub status for tag:c137a699-8ffd-44e3-b84e-ffc8358c9c76 - {'path': '/', 'tag': 'c137a699-8ffd-44e3-b84e-ffc8358c9c76', 'options': 'recursive,force'} 2024-08-01T04:24:52.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:51 smithi160 ceph-mon[37224]: pgmap v264: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:24:52.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:51 smithi163 ceph-mon[37383]: pgmap v264: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:24:52.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:51 smithi003 ceph-mon[32262]: pgmap v264: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:24:54.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:53 smithi160 ceph-mon[37224]: pgmap v265: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:24:54.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:53 smithi163 ceph-mon[37383]: pgmap v265: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:24:54.350 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:53 smithi003 ceph-mon[32262]: pgmap v265: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:24:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:55 smithi160 ceph-mon[37224]: pgmap v266: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:24:56.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:55 smithi163 ceph-mon[37383]: pgmap v266: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:24:56.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:55 smithi003 ceph-mon[32262]: pgmap v266: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:24:57.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:56 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:24:57.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:56 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:24:57.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:56 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:24:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:57 smithi160 ceph-mon[37224]: pgmap v267: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:24:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:57 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:24:58.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:57 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:24:58.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:57 smithi163 ceph-mon[37383]: pgmap v267: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:24:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:57 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:24:58.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:57 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:24:58.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:57 smithi003 ceph-mon[32262]: pgmap v267: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:24:58.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:57 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:24:58.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:57 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:25:00.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:24:59 smithi160 ceph-mon[37224]: pgmap v268: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:00.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:24:59 smithi163 ceph-mon[37383]: pgmap v268: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:00.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:24:59 smithi003 ceph-mon[32262]: pgmap v268: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:01 smithi160 ceph-mon[37224]: pgmap v269: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-01T04:25:02.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:01 smithi163 ceph-mon[37383]: pgmap v269: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-01T04:25:02.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:01 smithi003 ceph-mon[32262]: pgmap v269: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-01T04:25:04.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:03 smithi160 ceph-mon[37224]: pgmap v270: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:25:04.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:03 smithi163 ceph-mon[37383]: pgmap v270: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:25:04.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:03 smithi003 ceph-mon[32262]: pgmap v270: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:25:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:05 smithi160 ceph-mon[37224]: pgmap v271: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:25:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:05 smithi163 ceph-mon[37383]: pgmap v271: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:25:06.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:05 smithi003 ceph-mon[32262]: pgmap v271: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:25:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:07 smithi003 ceph-mon[32262]: pgmap v272: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:25:08.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:07 smithi003 ceph-mon[32262]: osdmap e96: 12 total, 12 up, 12 in 2024-08-01T04:25:08.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:07 smithi163 ceph-mon[37383]: pgmap v272: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:25:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:07 smithi163 ceph-mon[37383]: osdmap e96: 12 total, 12 up, 12 in 2024-08-01T04:25:08.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:07 smithi160 ceph-mon[37224]: pgmap v272: 97 pgs: 97 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:25:08.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:07 smithi160 ceph-mon[37224]: osdmap e96: 12 total, 12 up, 12 in 2024-08-01T04:25:10.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:09 smithi163 ceph-mon[37383]: pgmap v274: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:25:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:09 smithi003 ceph-mon[32262]: pgmap v274: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:25:10.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:09 smithi160 ceph-mon[37224]: pgmap v274: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 27 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:25:12.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:11 smithi163 ceph-mon[37383]: pgmap v275: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:25:12.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:11 smithi003 ceph-mon[32262]: pgmap v275: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:25:12.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:11 smithi160 ceph-mon[37224]: pgmap v275: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:25:14.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:13 smithi163 ceph-mon[37383]: pgmap v276: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:25:14.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:13 smithi003 ceph-mon[32262]: pgmap v276: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:25:14.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:13 smithi160 ceph-mon[37224]: pgmap v276: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:25:15.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:14 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:25:15.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:14 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:25:15.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:14 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:25:16.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:15 smithi163 ceph-mon[37383]: pgmap v277: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:25:16.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:15 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:25:16.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:15 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:25:16.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:15 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:25:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:15 smithi003 ceph-mon[32262]: pgmap v277: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:25:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:15 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:25:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:15 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:25:16.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:15 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:25:16.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:15 smithi160 ceph-mon[37224]: pgmap v277: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:25:16.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:15 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:25:16.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:15 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:25:16.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:15 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:25:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:17 smithi163 ceph-mon[37383]: pgmap v278: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:25:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:17 smithi003 ceph-mon[32262]: pgmap v278: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:25:18.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:17 smithi160 ceph-mon[37224]: pgmap v278: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:25:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:19 smithi163 ceph-mon[37383]: pgmap v279: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 182 B/s wr, 7 op/s 2024-08-01T04:25:20.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:19 smithi163 ceph-mon[37383]: osdmap e97: 12 total, 12 up, 12 in 2024-08-01T04:25:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:19 smithi003 ceph-mon[32262]: pgmap v279: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 182 B/s wr, 7 op/s 2024-08-01T04:25:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:19 smithi003 ceph-mon[32262]: osdmap e97: 12 total, 12 up, 12 in 2024-08-01T04:25:20.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:19 smithi160 ceph-mon[37224]: pgmap v279: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 182 B/s wr, 7 op/s 2024-08-01T04:25:20.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:19 smithi160 ceph-mon[37224]: osdmap e97: 12 total, 12 up, 12 in 2024-08-01T04:25:20.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:25:21.042 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:22.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:21 smithi003 ceph-mon[32262]: pgmap v281: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:22.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.115+0000 7f2387400640 1 -- 172.21.15.3:0/4281440314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380112760 msgr2=0x7f2380114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.115+0000 7f2387400640 1 --2- 172.21.15.3:0/4281440314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380112760 0x7f2380114b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ca0 tx=0x7f237802a8c0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 -- 172.21.15.3:0/4281440314 shutdown_connections 2024-08-01T04:25:22.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 --2- 172.21.15.3:0/4281440314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380112760 0x7f2380114b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 --2- 172.21.15.3:0/4281440314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 0x7f2380112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 --2- 172.21.15.3:0/4281440314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380076450 0x7f2380076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 -- 172.21.15.3:0/4281440314 >> 172.21.15.3:0/4281440314 conn(0x7f23800ff7c0 msgr2=0x7f2380101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 -- 172.21.15.3:0/4281440314 shutdown_connections 2024-08-01T04:25:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.116+0000 7f2387400640 1 -- 172.21.15.3:0/4281440314 wait complete. 2024-08-01T04:25:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.117+0000 7f2387400640 1 Processor -- start 2024-08-01T04:25:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.117+0000 7f2387400640 1 -- start start 2024-08-01T04:25:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2387400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380076450 0x7f238019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2387400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 0x7f238019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2387400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380112760 0x7f23801a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2387400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23801182f0 con 0x7f2380076d70 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2387400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2380118170 con 0x7f2380112760 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2387400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2380118470 con 0x7f2380076450 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f237fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 0x7f238019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2385400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380112760 0x7f23801a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f2384a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380076450 0x7f238019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f237fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 0x7f238019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48030/0 (socket says 172.21.15.3:48030) 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.118+0000 7f237fe00640 1 -- 172.21.15.3:0/2370395439 learned_addr learned my addr 172.21.15.3:0/2370395439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f2385400640 1 -- 172.21.15.3:0/2370395439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380076450 msgr2=0x7f238019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f2385400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380076450 0x7f238019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f2385400640 1 -- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 msgr2=0x7f238019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f2385400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 0x7f238019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f2385400640 1 -- 172.21.15.3:0/2370395439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23801a36e0 con 0x7f2380112760 2024-08-01T04:25:22.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f2385400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380112760 0x7f23801a3000 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f237802aeb0 tx=0x7f2378003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.119+0000 7f237d600640 1 -- 172.21.15.3:0/2370395439 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378031650 con 0x7f2380112760 2024-08-01T04:25:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.120+0000 7f237d600640 1 -- 172.21.15.3:0/2370395439 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23780317f0 con 0x7f2380112760 2024-08-01T04:25:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.120+0000 7f237d600640 1 -- 172.21.15.3:0/2370395439 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237802f990 con 0x7f2380112760 2024-08-01T04:25:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.120+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23801a3970 con 0x7f2380112760 2024-08-01T04:25:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.120+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23801a3e30 con 0x7f2380112760 2024-08-01T04:25:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.123+0000 7f237d600640 1 -- 172.21.15.3:0/2370395439 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f237802fb30 con 0x7f2380112760 2024-08-01T04:25:22.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.124+0000 7f237d600640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23640a2d70 0x7f23640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.125+0000 7f237d600640 1 -- 172.21.15.3:0/2370395439 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f23780f0760 con 0x7f2380112760 2024-08-01T04:25:22.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.125+0000 7f2384a00640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23640a2d70 0x7f23640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.126+0000 7f2384a00640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23640a2d70 0x7f23640a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2370009bd0 tx=0x7f2370005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.133+0000 7f2389ea3740 1 Processor -- start 2024-08-01T04:25:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.133+0000 7f2389ea3740 1 -- start start 2024-08-01T04:25:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60ccbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2389ea3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55be60c31bf0 con 0x55be60ccbb90 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2389ea3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55be60bc9bd0 con 0x55be60ccc4b0 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2389ea3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55be60c2dbb0 con 0x55be60cd6d70 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2384a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60ccbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2384a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60ccbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48034/0 (socket says 172.21.15.3:48034) 2024-08-01T04:25:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2384a00640 1 -- 172.21.15.3:0/3803521381 learned_addr learned my addr 172.21.15.3:0/3803521381 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:22.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f237fe00640 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.134+0000 7f2385400640 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.135+0000 7f237fe00640 1 -- 172.21.15.3:0/3803521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 msgr2=0x55be60cd9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.135+0000 7f237fe00640 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.135+0000 7f237fe00640 1 -- 172.21.15.3:0/3803521381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 msgr2=0x55be60ccbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.135+0000 7f237fe00640 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60ccbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.135+0000 7f237fe00640 1 -- 172.21.15.3:0/3803521381 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be60bd08a0 con 0x55be60ccc4b0 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.135+0000 7f237fe00640 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd6830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2368009c00 tx=0x7f2368011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ea28b35e83f57e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.136+0000 7f2375c00640 1 -- 172.21.15.3:0/3803521381 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004230 con 0x55be60ccc4b0 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.136+0000 7f2375c00640 1 -- 172.21.15.3:0/3803521381 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23680043d0 con 0x55be60ccc4b0 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.136+0000 7f2375c00640 1 -- 172.21.15.3:0/3803521381 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004d70 con 0x55be60ccc4b0 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.136+0000 7f2389ea3740 1 -- 172.21.15.3:0/3803521381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 msgr2=0x55be60cd6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.136+0000 7f2389ea3740 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd6830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2368009c00 tx=0x7f2368011810 comp rx=0 tx=0).stop 2024-08-01T04:25:22.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 -- 172.21.15.3:0/3803521381 shutdown_connections 2024-08-01T04:25:22.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd6830 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 --2- 172.21.15.3:0/3803521381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60ccbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 -- 172.21.15.3:0/3803521381 >> 172.21.15.3:0/3803521381 conn(0x55be60cc8120 msgr2=0x55be60cc8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:22.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 -- 172.21.15.3:0/3803521381 shutdown_connections 2024-08-01T04:25:22.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 -- 172.21.15.3:0/3803521381 wait complete. 2024-08-01T04:25:22.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.137+0000 7f2389ea3740 1 -- start start 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 Processor -- start 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55be60cd9940 con 0x55be60ccbb90 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55be60becd40 con 0x55be60ccc4b0 2024-08-01T04:25:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55be60cd9ac0 con 0x55be60cd6d70 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.138+0000 7f2389ea3740 10 client.0 _ll_register_callbacks cb 0x55be60bb5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f2384a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33848/0 (socket says 172.21.15.3:33848) 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 -- 172.21.15.3:0/1119612408 learned_addr learned my addr 172.21.15.3:0/1119612408 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f2384a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48044/0 (socket says 172.21.15.3:48044) 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f2385400640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 -- 172.21.15.3:0/1119612408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 msgr2=0x55be60cd46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 -- 172.21.15.3:0/1119612408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 msgr2=0x55be60cd3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f237fe00640 1 -- 172.21.15.3:0/1119612408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be609a33c0 con 0x55be60ccc4b0 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.139+0000 7f2384a00640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f237fe00640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd4160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2368004990 tx=0x7f2368019a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6656c397fb67396 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:22.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f236e000640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f236e000640 1 -- 172.21.15.3:0/1119612408 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004f00 con 0x55be60ccc4b0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23680043d0 con 0x55be60ccc4b0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f236e000640 1 -- 172.21.15.3:0/1119612408 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f23680050a0 con 0x55be60ccc4b0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f236e000640 1 -- 172.21.15.3:0/1119612408 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2368032050 con 0x55be60ccc4b0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f2389ea3740 10 client.26267 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.140+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be60ce55c0 con 0x55be60ccc4b0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f236e000640 1 -- 172.21.15.3:0/1119612408 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23680043d0 con 0x55be60ccc4b0 2024-08-01T04:25:22.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f236e000640 1 -- 172.21.15.3:0/1119612408 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2368018070 con 0x55be60ccc4b0 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f2389ea3740 10 client.26267 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f2389ea3740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f2389ea3740 20 client.26267 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f2389ea3740 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60ce6970 0x55be60ce8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f2389ea3740 4 client.26267 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.141+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be60c3f160 con 0x55be60ce6970 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.142+0000 7f2384a00640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60ce6970 0x55be60ce8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.142+0000 7f2384a00640 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60ce6970 0x55be60ce8d60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f476c350704a73aa server_cookie=8466dcd3e8be62fd in_seq=0 out_seq=0 2024-08-01T04:25:22.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.142+0000 7f236e000640 10 client.26267 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.143+0000 7f236e000640 1 -- 172.21.15.3:0/1119612408 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be60c3f160 con 0x55be60ce6970 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.143+0000 7f236e000640 10 client.26267 handle_command_reply: tid=0 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.143+0000 7f2389ea3740 1 client.26267 shutdown 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 msgr2=0x55be60cd4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd4160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2368004990 tx=0x7f2368019a50 comp rx=0 tx=0).stop 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 shutdown_connections 2024-08-01T04:25:22.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60ce6970 0x55be60ce8d60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cd46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cd4160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 --2- 172.21.15.3:0/1119612408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f236e000640 0 client.26267 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 >> 172.21.15.3:0/1119612408 conn(0x55be60cc8120 msgr2=0x55be60cc7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 shutdown_connections 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 1 -- 172.21.15.3:0/1119612408 wait complete. 2024-08-01T04:25:22.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.144+0000 7f2389ea3740 20 client.26267 trim_cache size 0 max 16384 2024-08-01T04:25:22.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 Processor -- start 2024-08-01T04:25:22.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 -- start start 2024-08-01T04:25:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60ccbb90 0x55be60cd2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 0x55be60cd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60cd6d70 0x55be60cee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b6600 con 0x55be60ccc4b0 2024-08-01T04:25:22.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b3ed0 con 0x55be60cd6d70 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.164+0000 7f2389ea3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b1d40 con 0x55be60ccbb90 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 0x55be60cd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 0x55be60cd2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48058/0 (socket says 172.21.15.3:48058) 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 -- 172.21.15.3:0/4261133171 learned_addr learned my addr 172.21.15.3:0/4261133171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f2384a00640 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60ccbb90 0x55be60cd2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f2385400640 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60cd6d70 0x55be60cee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 -- 172.21.15.3:0/4261133171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60ccbb90 msgr2=0x55be60cd2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60ccbb90 0x55be60cd2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 -- 172.21.15.3:0/4261133171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60cd6d70 msgr2=0x55be60cee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60cd6d70 0x55be60cee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f237fe00640 1 -- 172.21.15.3:0/4261133171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be60ce91c0 con 0x55be60ccc4b0 2024-08-01T04:25:22.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.165+0000 7f2385400640 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60cd6d70 0x55be60cee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:25:22.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.166+0000 7f237fe00640 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 0x55be60cd2a70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f236801c040 tx=0x7f2368011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd56bd6c1e051cdd server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:22.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.166+0000 7f2375200640 1 -- 172.21.15.3:0/4261133171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368019ac0 con 0x55be60ccc4b0 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.166+0000 7f2375200640 1 -- 172.21.15.3:0/4261133171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2368029070 con 0x55be60ccc4b0 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.166+0000 7f2375200640 1 -- 172.21.15.3:0/4261133171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004570 con 0x55be60ccc4b0 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 -- 172.21.15.3:0/4261133171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 msgr2=0x55be60cd2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 0x55be60cd2a70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f236801c040 tx=0x7f2368011810 comp rx=0 tx=0).stop 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 -- 172.21.15.3:0/4261133171 shutdown_connections 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60cd6d70 0x55be60cee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccc4b0 0x55be60cd2a70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 --2- 172.21.15.3:0/4261133171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60ccbb90 0x55be60cd2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 -- 172.21.15.3:0/4261133171 >> 172.21.15.3:0/4261133171 conn(0x55be60cc8120 msgr2=0x55be60cc2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 -- 172.21.15.3:0/4261133171 shutdown_connections 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.167+0000 7f2389ea3740 1 -- 172.21.15.3:0/4261133171 wait complete. 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.168+0000 7f2389ea3740 1 -- start start 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.168+0000 7f2389ea3740 1 Processor -- start 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cdf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b2390 con 0x55be60ccbb90 2024-08-01T04:25:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b3800 con 0x55be60ccc4b0 2024-08-01T04:25:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f23801b2a60 con 0x55be60cd6d70 2024-08-01T04:25:22.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2389ea3740 10 client.0 _ll_register_callbacks cb 0x7f2370009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:22.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2384a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f237fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2384a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48060/0 (socket says 172.21.15.3:48060) 2024-08-01T04:25:22.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2384a00640 1 -- 172.21.15.3:0/692740977 learned_addr learned my addr 172.21.15.3:0/692740977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:22.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2385400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cdf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.169+0000 7f2385400640 1 -- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 msgr2=0x55be60cded10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2385400640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cded10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2385400640 1 -- 172.21.15.3:0/692740977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 msgr2=0x55be60cd17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2385400640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2385400640 1 -- 172.21.15.3:0/692740977 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be60ccfed0 con 0x55be60cd6d70 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f237fe00640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2384a00640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2385400640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cdf2b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23780edef0 tx=0x7f2378004850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86fd3de09c0a28c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:22.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f236e000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f236e000640 1 -- 172.21.15.3:0/692740977 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23780c1070 con 0x55be60cd6d70 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2368029070 con 0x55be60cd6d70 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f236e000640 1 -- 172.21.15.3:0/692740977 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f2378009890 con 0x55be60cd6d70 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f236e000640 1 -- 172.21.15.3:0/692740977 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23780c5930 con 0x55be60cd6d70 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.170+0000 7f2389ea3740 10 client.26248 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be60d50680 con 0x55be60cd6d70 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f236e000640 1 -- 172.21.15.3:0/692740977 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2368029070 con 0x55be60cd6d70 2024-08-01T04:25:22.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f236e000640 1 -- 172.21.15.3:0/692740977 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f23780c54a0 con 0x55be60cd6d70 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 10 client.26248 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 20 client.26248 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60d50d50 0x55be60d53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 4 client.26248 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be60c39670 con 0x55be60d50d50 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.171+0000 7f2384a00640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60d50d50 0x55be60d53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:22.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.172+0000 7f2384a00640 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60d50d50 0x55be60d53170 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10aae953c6cdc108 server_cookie=794dc9b7deb73bdc in_seq=0 out_seq=0 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.172+0000 7f236e000640 10 client.26248 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.172+0000 7f236e000640 1 -- 172.21.15.3:0/692740977 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be60c39670 con 0x55be60d50d50 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.172+0000 7f236e000640 10 client.26248 handle_command_reply: tid=0 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.172+0000 7f2389ea3740 1 client.26248 shutdown 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 msgr2=0x55be60cdf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cdf2b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23780edef0 tx=0x7f2378004850 comp rx=0 tx=0).stop 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 shutdown_connections 2024-08-01T04:25:22.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55be60d50d50 0x55be60d53170 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be60cd6d70 0x55be60cdf2b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55be60ccc4b0 0x55be60cded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 --2- 172.21.15.3:0/692740977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55be60ccbb90 0x55be60cd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 >> 172.21.15.3:0/692740977 conn(0x55be60cc8120 msgr2=0x55be60cd24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f236e000640 0 client.26248 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 shutdown_connections 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 1 -- 172.21.15.3:0/692740977 wait complete. 2024-08-01T04:25:22.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.173+0000 7f2389ea3740 20 client.26248 trim_cache size 0 max 16384 2024-08-01T04:25:22.182 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:25:22.182 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:25:22.182 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:25:22.182 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:25:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23640a2d70 msgr2=0x7f23640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23640a2d70 0x7f23640a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2370009bd0 tx=0x7f2370005f90 comp rx=0 tx=0).stop 2024-08-01T04:25:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380112760 msgr2=0x7f23801a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:22.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380112760 0x7f23801a3000 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f237802aeb0 tx=0x7f2378003f10 comp rx=0 tx=0).stop 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 shutdown_connections 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2380112760 0x7f23801a3000 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2380076d70 0x7f238019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f23640a2d70 0x7f23640a5230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 --2- 172.21.15.3:0/2370395439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2380076450 0x7f238019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 >> 172.21.15.3:0/2370395439 conn(0x7f23800ff7c0 msgr2=0x7f2380101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 shutdown_connections 2024-08-01T04:25:22.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:22.174+0000 7f2387400640 1 -- 172.21.15.3:0/2370395439 wait complete. 2024-08-01T04:25:22.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:21 smithi163 ceph-mon[37383]: pgmap v281: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:22.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:21 smithi160 ceph-mon[37224]: pgmap v281: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:23.191 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:25:23.192 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:25:23.678 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:24.100 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:23 smithi003 ceph-mon[32262]: pgmap v282: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:24.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:23 smithi163 ceph-mon[37383]: pgmap v282: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:24.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 -- 172.21.15.3:0/545584538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898103900 msgr2=0x7f4898103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 --2- 172.21.15.3:0/545584538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898103900 0x7f4898103d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4880009890 tx=0x7f488002a360 comp rx=0 tx=0).stop 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 -- 172.21.15.3:0/545584538 shutdown_connections 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 --2- 172.21.15.3:0/545584538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48981042a0 0x7f48981107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 --2- 172.21.15.3:0/545584538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898103900 0x7f4898103d60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 --2- 172.21.15.3:0/545584538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4898109900 0x7f4898109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 -- 172.21.15.3:0/545584538 >> 172.21.15.3:0/545584538 conn(0x7f48980ff640 msgr2=0x7f4898101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:24.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 -- 172.21.15.3:0/545584538 shutdown_connections 2024-08-01T04:25:24.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.391+0000 7f489ce00640 1 -- 172.21.15.3:0/545584538 wait complete. 2024-08-01T04:25:24.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.392+0000 7f489ce00640 1 Processor -- start 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.392+0000 7f489ce00640 1 -- start start 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f489ce00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4898103900 0x7f489819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f489ce00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 0x7f489819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f489ce00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898109900 0x7f489819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f489ce00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4898113d00 con 0x7f4898103900 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f489ce00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4898113b80 con 0x7f48981042a0 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f489ce00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4898113e80 con 0x7f4898109900 2024-08-01T04:25:24.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f488ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 0x7f489819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f4897400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4898103900 0x7f489819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f488ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 0x7f489819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33878/0 (socket says 172.21.15.3:33878) 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f488ea00640 1 -- 172.21.15.3:0/2333211822 learned_addr learned my addr 172.21.15.3:0/2333211822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.393+0000 7f4897e00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898109900 0x7f489819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.394+0000 7f488ea00640 1 -- 172.21.15.3:0/2333211822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898109900 msgr2=0x7f489819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.394+0000 7f488ea00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898109900 0x7f489819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.394+0000 7f488ea00640 1 -- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4898103900 msgr2=0x7f489819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:24.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.394+0000 7f488ea00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4898103900 0x7f489819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.394+0000 7f488ea00640 1 -- 172.21.15.3:0/2333211822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489819f1f0 con 0x7f48981042a0 2024-08-01T04:25:24.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.394+0000 7f488ea00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 0x7f489819a780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4880000c00 tx=0x7f4880033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:24.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.395+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488003f020 con 0x7f48981042a0 2024-08-01T04:25:24.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.395+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489819f480 con 0x7f48981042a0 2024-08-01T04:25:24.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.395+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4880032770 con 0x7f48981042a0 2024-08-01T04:25:24.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.395+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880037040 con 0x7f48981042a0 2024-08-01T04:25:24.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.395+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48981a6dc0 con 0x7f48981042a0 2024-08-01T04:25:24.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.396+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4898105040 con 0x7f48981042a0 2024-08-01T04:25:24.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.398+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4880042020 con 0x7f48981042a0 2024-08-01T04:25:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:23 smithi160 ceph-mon[37224]: pgmap v282: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:24.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.400+0000 7f4894c00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f487c0a2cf0 0x7f487c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:24.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.401+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f48800ef3d0 con 0x7f48981042a0 2024-08-01T04:25:24.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.401+0000 7f4897400640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f487c0a2cf0 0x7f487c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:24.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.402+0000 7f4897400640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f487c0a2cf0 0x7f487c0a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4888009b50 tx=0x7f4888005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:24.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.402+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4880049020 con 0x7f48981042a0 2024-08-01T04:25:24.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.637+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f489810bea0 con 0x7f48981042a0 2024-08-01T04:25:24.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.638+0000 7f4894c00640 1 -- 172.21.15.3:0/2333211822 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f48800bc030 con 0x7f48981042a0 2024-08-01T04:25:24.641 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:25:24.642 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:25:24.642 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:25:24.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.641+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f487c0a2cf0 msgr2=0x7f487c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:24.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.641+0000 7f489ce00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f487c0a2cf0 0x7f487c0a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4888009b50 tx=0x7f4888005f90 comp rx=0 tx=0).stop 2024-08-01T04:25:24.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 msgr2=0x7f489819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:24.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 0x7f489819a780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4880000c00 tx=0x7f4880033d10 comp rx=0 tx=0).stop 2024-08-01T04:25:24.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 shutdown_connections 2024-08-01T04:25:24.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f487c0a2cf0 0x7f487c0a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4898109900 0x7f489819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48981042a0 0x7f489819a780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 --2- 172.21.15.3:0/2333211822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4898103900 0x7f489819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:24.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 >> 172.21.15.3:0/2333211822 conn(0x7f48980ff640 msgr2=0x7f48980fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:24.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 shutdown_connections 2024-08-01T04:25:24.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:24.642+0000 7f489ce00640 1 -- 172.21.15.3:0/2333211822 wait complete. 2024-08-01T04:25:25.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:25:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:25 smithi160 ceph-mon[37224]: pgmap v283: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:25.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2333211822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:25:25.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:25 smithi003 ceph-mon[32262]: pgmap v283: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:25.977 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2333211822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:25:26.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:25 smithi163 ceph-mon[37383]: pgmap v283: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:25:26.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2333211822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:25:26.086 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 -- 172.21.15.3:0/197860734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf81078b0 msgr2=0x7efcf8107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 --2- 172.21.15.3:0/197860734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf81078b0 0x7efcf8107c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efcec009930 tx=0x7efcec02a420 comp rx=0 tx=0).stop 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 -- 172.21.15.3:0/197860734 shutdown_connections 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 --2- 172.21.15.3:0/197860734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf8102250 0x7efcf810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 --2- 172.21.15.3:0/197860734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81018b0 0x7efcf8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 --2- 172.21.15.3:0/197860734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf81078b0 0x7efcf8107c90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 -- 172.21.15.3:0/197860734 >> 172.21.15.3:0/197860734 conn(0x7efcf80fd5f0 msgr2=0x7efcf80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 -- 172.21.15.3:0/197860734 shutdown_connections 2024-08-01T04:25:27.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.097+0000 7efd00400640 1 -- 172.21.15.3:0/197860734 wait complete. 2024-08-01T04:25:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.098+0000 7efd00400640 1 Processor -- start 2024-08-01T04:25:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.098+0000 7efd00400640 1 -- start start 2024-08-01T04:25:27.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efd00400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf81018b0 0x7efcf819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efd00400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 0x7efcf819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efd00400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81078b0 0x7efcf819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efd00400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8111cc0 con 0x7efcf81078b0 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efd00400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8111b40 con 0x7efcf81018b0 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efd00400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8111e40 con 0x7efcf8102250 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efcfd000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 0x7efcf819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efcfe400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81078b0 0x7efcf819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efcfda00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf81018b0 0x7efcf819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efcfd000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 0x7efcf819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:44852/0 (socket says 172.21.15.3:44852) 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.099+0000 7efcfd000640 1 -- 172.21.15.3:0/2378901959 learned_addr learned my addr 172.21.15.3:0/2378901959 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfd000640 1 -- 172.21.15.3:0/2378901959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf81018b0 msgr2=0x7efcf819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfd000640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf81018b0 0x7efcf819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfd000640 1 -- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81078b0 msgr2=0x7efcf819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfd000640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81078b0 0x7efcf819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfd000640 1 -- 172.21.15.3:0/2378901959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf819f3b0 con 0x7efcf8102250 2024-08-01T04:25:27.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfda00640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf81018b0 0x7efcf819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:27.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfe400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81078b0 0x7efcf819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:27.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcfd000640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 0x7efcf819a940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9f0 tx=0x7efce400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:27.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcf6000640 1 -- 172.21.15.3:0/2378901959 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce400cd80 con 0x7efcf8102250 2024-08-01T04:25:27.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.100+0000 7efcf6000640 1 -- 172.21.15.3:0/2378901959 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce4010910 con 0x7efcf8102250 2024-08-01T04:25:27.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.101+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf819f6a0 con 0x7efcf8102250 2024-08-01T04:25:27.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.101+0000 7efcf6000640 1 -- 172.21.15.3:0/2378901959 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4018990 con 0x7efcf8102250 2024-08-01T04:25:27.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.101+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81a6f80 con 0x7efcf8102250 2024-08-01T04:25:27.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.104+0000 7efd0305b740 1 Processor -- start 2024-08-01T04:25:27.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.104+0000 7efd0305b740 1 -- start start 2024-08-01T04:25:27.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.104+0000 7efcf6000640 1 -- 172.21.15.3:0/2378901959 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7efce4010ac0 con 0x7efcf8102250 2024-08-01T04:25:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.104+0000 7efd0305b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93e4f0 0x55fb2e93e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.104+0000 7efd0305b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e949190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efd0305b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e94bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efd0305b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fb2e8a3bf0 con 0x55fb2e93ee10 2024-08-01T04:25:27.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efd0305b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fb2e83bb20 con 0x55fb2e93e4f0 2024-08-01T04:25:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efd0305b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fb2e89fbb0 con 0x55fb2e9496d0 2024-08-01T04:25:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e949190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e949190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48114/0 (socket says 172.21.15.3:48114) 2024-08-01T04:25:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 -- 172.21.15.3:0/3646696840 learned_addr learned my addr 172.21.15.3:0/3646696840 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfe400640 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93e4f0 0x55fb2e93e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfd000640 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e94bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 -- 172.21.15.3:0/3646696840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 msgr2=0x55fb2e94bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e94bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 -- 172.21.15.3:0/3646696840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93e4f0 msgr2=0x55fb2e93e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93e4f0 0x55fb2e93e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfda00640 1 -- 172.21.15.3:0/3646696840 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2e8427f0 con 0x55fb2e93ee10 2024-08-01T04:25:27.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.105+0000 7efcfd000640 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e94bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:25:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.106+0000 7efcfda00640 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e949190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcec009930 tx=0x7efcec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868bbdcae686133c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.106+0000 7efceaa00640 1 -- 172.21.15.3:0/3646696840 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec035040 con 0x55fb2e93ee10 2024-08-01T04:25:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.106+0000 7efcf6000640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7efce00a2d70 0x7efce00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.106+0000 7efceaa00640 1 -- 172.21.15.3:0/3646696840 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcec004760 con 0x55fb2e93ee10 2024-08-01T04:25:27.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efcf6000640 1 -- 172.21.15.3:0/2378901959 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7efce40cf790 con 0x7efcf8102250 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efcfe400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7efce00a2d70 0x7efce00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efceaa00640 1 -- 172.21.15.3:0/3646696840 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0038e0 con 0x55fb2e93ee10 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 -- 172.21.15.3:0/3646696840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 msgr2=0x55fb2e949190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e949190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efcec009930 tx=0x7efcec000bc0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 -- 172.21.15.3:0/3646696840 shutdown_connections 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e94bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e949190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 --2- 172.21.15.3:0/3646696840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93e4f0 0x55fb2e93e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.107+0000 7efd0305b740 1 -- 172.21.15.3:0/3646696840 >> 172.21.15.3:0/3646696840 conn(0x55fb2e93aa80 msgr2=0x55fb2e93ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.108+0000 7efcfe400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7efce00a2d70 0x7efce00a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009fd0 tx=0x7efcf0009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.108+0000 7efd0305b740 1 -- 172.21.15.3:0/3646696840 shutdown_connections 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.108+0000 7efd0305b740 1 -- 172.21.15.3:0/3646696840 wait complete. 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.108+0000 7efd0305b740 1 -- start start 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.108+0000 7efd0305b740 1 Processor -- start 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93ee10 0x55fb2e9469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e946f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fb2e94c2f0 con 0x55fb2e93e4f0 2024-08-01T04:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55fb2e85ecf0 con 0x55fb2e9496d0 2024-08-01T04:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55fb2e94c470 con 0x55fb2e93ee10 2024-08-01T04:25:27.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efd0305b740 10 client.0 _ll_register_callbacks cb 0x55fb2e84ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efcfda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efcfda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48120/0 (socket says 172.21.15.3:48120) 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efcfda00640 1 -- 172.21.15.3:0/1387541538 learned_addr learned my addr 172.21.15.3:0/1387541538 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efcfd000640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93ee10 0x55fb2e9469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.109+0000 7efcfe400640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e946f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcfda00640 1 -- 172.21.15.3:0/1387541538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93ee10 msgr2=0x55fb2e9469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcfda00640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93ee10 0x55fb2e9469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcfda00640 1 -- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 msgr2=0x55fb2e946f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcfda00640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e946f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcfda00640 1 -- 172.21.15.3:0/1387541538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2e6153c0 con 0x55fb2e93e4f0 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcfda00640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9464b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efcec009900 tx=0x7efcec004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac61b96756224c6e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcdf400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcdf400640 1 -- 172.21.15.3:0/1387541538 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec053060 con 0x55fb2e93e4f0 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcec004760 con 0x55fb2e93e4f0 2024-08-01T04:25:27.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.110+0000 7efcdf400640 1 -- 172.21.15.3:0/1387541538 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7efcec005130 con 0x55fb2e93e4f0 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efcdf400640 1 -- 172.21.15.3:0/1387541538 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcec035050 con 0x55fb2e93e4f0 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 10 client.16104 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb2e957f20 con 0x55fb2e93e4f0 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efcdf400640 1 -- 172.21.15.3:0/1387541538 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcec004760 con 0x55fb2e93e4f0 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efcdf400640 1 -- 172.21.15.3:0/1387541538 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7efcec0315b0 con 0x55fb2e93e4f0 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 10 client.16104 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 10 client.16104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:25:27.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 20 client.16104 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e959270 0x55fb2e95b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 4 client.16104 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.111+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb2e8b1160 con 0x55fb2e959270 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.112+0000 7efcfd000640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e959270 0x55fb2e95b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.112+0000 7efcfd000640 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e959270 0x55fb2e95b660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d90bc777c4c4cb server_cookie=a8c6414bbcbb3ca2 in_seq=0 out_seq=0 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.112+0000 7efcdf400640 10 client.16104 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:25:27.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.113+0000 7efcdf400640 1 -- 172.21.15.3:0/1387541538 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb2e8b1160 con 0x55fb2e959270 2024-08-01T04:25:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.113+0000 7efcdf400640 10 client.16104 handle_command_reply: tid=0 2024-08-01T04:25:27.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.113+0000 7efd0305b740 1 client.16104 shutdown 2024-08-01T04:25:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 msgr2=0x55fb2e9464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9464b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efcec009900 tx=0x7efcec004b40 comp rx=0 tx=0).stop 2024-08-01T04:25:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 shutdown_connections 2024-08-01T04:25:27.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e946f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e959270 0x55fb2e95b660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93ee10 0x55fb2e9469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 --2- 172.21.15.3:0/1387541538 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9464b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efcdf400640 0 client.16104 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 >> 172.21.15.3:0/1387541538 conn(0x55fb2e93aa80 msgr2=0x55fb2e939c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:27.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 shutdown_connections 2024-08-01T04:25:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 1 -- 172.21.15.3:0/1387541538 wait complete. 2024-08-01T04:25:27.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.114+0000 7efd0305b740 20 client.16104 trim_cache size 0 max 16384 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.128+0000 7efd0305b740 1 Processor -- start 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 -- start start 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93e4f0 0x55fb2e943710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e943c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e944c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81b2320 con 0x55fb2e93ee10 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81afbf0 con 0x55fb2e9496d0 2024-08-01T04:25:27.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efd0305b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81ada60 con 0x55fb2e93e4f0 2024-08-01T04:25:27.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efcfd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e943c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efcfd000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e943c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48134/0 (socket says 172.21.15.3:48134) 2024-08-01T04:25:27.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efcfd000640 1 -- 172.21.15.3:0/4143264641 learned_addr learned my addr 172.21.15.3:0/4143264641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:27.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.129+0000 7efcfda00640 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93e4f0 0x55fb2e943710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfe400640 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e944c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfd000640 1 -- 172.21.15.3:0/4143264641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93e4f0 msgr2=0x55fb2e943710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfd000640 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93e4f0 0x55fb2e943710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfd000640 1 -- 172.21.15.3:0/4143264641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 msgr2=0x55fb2e944c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfd000640 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e944c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfd000640 1 -- 172.21.15.3:0/4143264641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2e93a8b0 con 0x55fb2e93ee10 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcfd000640 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e943c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9c0 tx=0x7efce4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55941493835813d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcea000640 1 -- 172.21.15.3:0/4143264641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4099d30 con 0x55fb2e93ee10 2024-08-01T04:25:27.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcea000640 1 -- 172.21.15.3:0/4143264641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce4099f90 con 0x55fb2e93ee10 2024-08-01T04:25:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.130+0000 7efcea000640 1 -- 172.21.15.3:0/4143264641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4018b30 con 0x55fb2e93ee10 2024-08-01T04:25:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 -- 172.21.15.3:0/4143264641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 msgr2=0x55fb2e943c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e943c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9c0 tx=0x7efce4000bc0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 -- 172.21.15.3:0/4143264641 shutdown_connections 2024-08-01T04:25:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e9496d0 0x55fb2e944c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93ee10 0x55fb2e943c50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 --2- 172.21.15.3:0/4143264641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e93e4f0 0x55fb2e943710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 -- 172.21.15.3:0/4143264641 >> 172.21.15.3:0/4143264641 conn(0x55fb2e93aa80 msgr2=0x55fb2e948530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:27.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 -- 172.21.15.3:0/4143264641 shutdown_connections 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.131+0000 7efd0305b740 1 -- 172.21.15.3:0/4143264641 wait complete. 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.132+0000 7efd0305b740 1 -- start start 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.132+0000 7efd0305b740 1 Processor -- start 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.132+0000 7efd0305b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efd0305b740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93ee10 0x55fb2e9bc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efd0305b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e95e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efd0305b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81ae0b0 con 0x55fb2e93e4f0 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efd0305b740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81af520 con 0x55fb2e93ee10 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efd0305b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81ae780 con 0x55fb2e9496d0 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efd0305b740 10 client.0 _ll_register_callbacks cb 0x7efcec004e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9bbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48148/0 (socket says 172.21.15.3:48148) 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 -- 172.21.15.3:0/4251314081 learned_addr learned my addr 172.21.15.3:0/4251314081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfd000640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93ee10 0x55fb2e9bc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfe400640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e95e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 -- 172.21.15.3:0/4251314081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 msgr2=0x55fb2e95e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e95e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 -- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93ee10 msgr2=0x55fb2e9bc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93ee10 0x55fb2e9bc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.133+0000 7efcfda00640 1 -- 172.21.15.3:0/4251314081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2e95ea30 con 0x55fb2e93e4f0 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efcfda00640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9bbd40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efcec031b70 tx=0x7efcec031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f6c31af52be005 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efcdf400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:27.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efcdf400640 1 -- 172.21.15.3:0/4251314081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec053070 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efce4099f90 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efcdf400640 1 -- 172.21.15.3:0/4251314081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7efcec005340 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efcdf400640 1 -- 172.21.15.3:0/4251314081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcec066050 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efd0305b740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.134+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb2e9c3600 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efcdf400640 1 -- 172.21.15.3:0/4251314081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce4099f90 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efcdf400640 1 -- 172.21.15.3:0/4251314081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7efcec04e470 con 0x55fb2e93e4f0 2024-08-01T04:25:27.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efd0305b740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efd0305b740 10 client.16110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efd0305b740 20 client.16110 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efd0305b740 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e9c3cc0 0x55fb2e9c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efd0305b740 4 client.16110 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb2e8ab670 con 0x55fb2e9c3cc0 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.135+0000 7efcfd000640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e9c3cc0 0x55fb2e9c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efcfd000640 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e9c3cc0 0x55fb2e9c60e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=423868050189eaf8 server_cookie=4aff533855654b5c in_seq=0 out_seq=0 2024-08-01T04:25:27.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efcdf400640 10 client.16110 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efcdf400640 1 -- 172.21.15.3:0/4251314081 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb2e8ab670 con 0x55fb2e9c3cc0 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efcdf400640 10 client.16110 handle_command_reply: tid=0 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efd0305b740 1 client.16110 shutdown 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 msgr2=0x55fb2e9bbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.136+0000 7efd0305b740 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9bbd40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efcec031b70 tx=0x7efcec031400 comp rx=0 tx=0).stop 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 shutdown_connections 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55fb2e9c3cc0 0x55fb2e9c60e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55fb2e9496d0 0x55fb2e95e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb2e93ee10 0x55fb2e9bc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 --2- 172.21.15.3:0/4251314081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fb2e93e4f0 0x55fb2e9bbd40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 >> 172.21.15.3:0/4251314081 conn(0x55fb2e93aa80 msgr2=0x55fb2e9393e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efcdf400640 0 client.16110 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 shutdown_connections 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 1 -- 172.21.15.3:0/4251314081 wait complete. 2024-08-01T04:25:27.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.137+0000 7efd0305b740 20 client.16110 trim_cache size 0 max 16384 2024-08-01T04:25:27.147 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:25:27.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7efce00a2d70 msgr2=0x7efce00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7efce00a2d70 0x7efce00a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009fd0 tx=0x7efcf0009500 comp rx=0 tx=0).stop 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 msgr2=0x7efcf819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 0x7efcf819a940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9f0 tx=0x7efce400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 shutdown_connections 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efcf81078b0 0x7efcf819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7efce00a2d70 0x7efce00a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf8102250 0x7efcf819a940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 --2- 172.21.15.3:0/2378901959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcf81018b0 0x7efcf819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 >> 172.21.15.3:0/2378901959 conn(0x7efcf80fd5f0 msgr2=0x7efcf810cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 shutdown_connections 2024-08-01T04:25:27.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:27.138+0000 7efd00400640 1 -- 172.21.15.3:0/2378901959 wait complete. 2024-08-01T04:25:27.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:27 smithi160 ceph-mon[37224]: pgmap v284: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:28.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:27 smithi163 ceph-mon[37383]: pgmap v284: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:28.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:27 smithi003 ceph-mon[32262]: pgmap v284: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:28.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:25:28.643 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:29.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 -- 172.21.15.3:0/459217517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 msgr2=0x7fdb241149b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 --2- 172.21.15.3:0/459217517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241149b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009ca0 tx=0x7fdb1c02a8c0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 -- 172.21.15.3:0/459217517 shutdown_connections 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 --2- 172.21.15.3:0/459217517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241149b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 --2- 172.21.15.3:0/459217517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076d70 0x7fdb24112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 --2- 172.21.15.3:0/459217517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb24076450 0x7fdb24076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 -- 172.21.15.3:0/459217517 >> 172.21.15.3:0/459217517 conn(0x7fdb240ff5e0 msgr2=0x7fdb24101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:29.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 -- 172.21.15.3:0/459217517 shutdown_connections 2024-08-01T04:25:29.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.285+0000 7fdb29000640 1 -- 172.21.15.3:0/459217517 wait complete. 2024-08-01T04:25:29.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.286+0000 7fdb29000640 1 Processor -- start 2024-08-01T04:25:29.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 -- start start 2024-08-01T04:25:29.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076450 0x7fdb2419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb24076d70 0x7fdb2419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24118280 con 0x7fdb241125c0 2024-08-01T04:25:29.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24118100 con 0x7fdb24076450 2024-08-01T04:25:29.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb29000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24118400 con 0x7fdb24076d70 2024-08-01T04:25:29.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48172/0 (socket says 172.21.15.3:48172) 2024-08-01T04:25:29.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb23e00640 1 -- 172.21.15.3:0/2817313099 learned_addr learned my addr 172.21.15.3:0/2817313099 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:29.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb23400640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076450 0x7fdb2419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.287+0000 7fdb22a00640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb24076d70 0x7fdb2419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23400640 1 -- 172.21.15.3:0/2817313099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb24076d70 msgr2=0x7fdb2419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23400640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb24076d70 0x7fdb2419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23400640 1 -- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 msgr2=0x7fdb241a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23400640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23400640 1 -- 172.21.15.3:0/2817313099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb241a3650 con 0x7fdb24076450 2024-08-01T04:25:29.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23e00640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:29.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.288+0000 7fdb23400640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076450 0x7fdb2419e6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e9f0 tx=0x7fdb1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:29.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.289+0000 7fdb1be00640 1 -- 172.21.15.3:0/2817313099 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400ccc0 con 0x7fdb24076450 2024-08-01T04:25:29.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.289+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb241a3940 con 0x7fdb24076450 2024-08-01T04:25:29.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.289+0000 7fdb1be00640 1 -- 172.21.15.3:0/2817313099 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb14005020 con 0x7fdb24076450 2024-08-01T04:25:29.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.289+0000 7fdb1be00640 1 -- 172.21.15.3:0/2817313099 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140049e0 con 0x7fdb24076450 2024-08-01T04:25:29.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.289+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241ab220 con 0x7fdb24076450 2024-08-01T04:25:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.292+0000 7fdb2d208740 1 Processor -- start 2024-08-01T04:25:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.292+0000 7fdb2d208740 1 -- start start 2024-08-01T04:25:29.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.292+0000 7fdb1be00640 1 -- 172.21.15.3:0/2817313099 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fdb14004c80 con 0x7fdb24076450 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd485380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd487cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb2d208740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e2bd3dfbf0 con 0x55e2bd4858c0 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb2d208740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e2bd377b20 con 0x55e2bd47a6e0 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb2d208740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e2bd3dbbb0 con 0x55e2bd47b000 2024-08-01T04:25:29.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd487cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb23400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd485380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33982/0 (socket says 172.21.15.3:33982) 2024-08-01T04:25:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb23e00640 1 -- 172.21.15.3:0/1413204764 learned_addr learned my addr 172.21.15.3:0/1413204764 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.293+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd487cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48176/0 (socket says 172.21.15.3:48176) 2024-08-01T04:25:29.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb23e00640 1 -- 172.21.15.3:0/1413204764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 msgr2=0x55e2bd485380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb23e00640 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd485380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb23e00640 1 -- 172.21.15.3:0/1413204764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 msgr2=0x55e2bd487cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb23e00640 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd487cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb23e00640 1 -- 172.21.15.3:0/1413204764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bd37e7f0 con 0x55e2bd47a6e0 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb22a00640 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd487cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb23e00640 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47aac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c033000 tx=0x7fdb1c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2c6a2a0b2aed1c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:29.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb1be00640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fdb080a2d70 0x7fdb080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb18c00640 1 -- 172.21.15.3:0/1413204764 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c003b70 con 0x55e2bd47a6e0 2024-08-01T04:25:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.294+0000 7fdb18c00640 1 -- 172.21.15.3:0/1413204764 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c036040 con 0x55e2bd47a6e0 2024-08-01T04:25:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.295+0000 7fdb18c00640 1 -- 172.21.15.3:0/1413204764 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c004630 con 0x55e2bd47a6e0 2024-08-01T04:25:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.295+0000 7fdb1be00640 1 -- 172.21.15.3:0/2817313099 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fdb140cf310 con 0x7fdb24076450 2024-08-01T04:25:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.295+0000 7fdb23400640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fdb080a2d70 0x7fdb080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.295+0000 7fdb2d208740 1 -- 172.21.15.3:0/1413204764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 msgr2=0x55e2bd47aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.295+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47aac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c033000 tx=0x7fdb1c005ef0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.295+0000 7fdb23400640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fdb080a2d70 0x7fdb080a5230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55e2bd477ad0 tx=0x7fdb140d2dc0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 -- 172.21.15.3:0/1413204764 shutdown_connections 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd487cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd485380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1413204764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47aac0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 -- 172.21.15.3:0/1413204764 >> 172.21.15.3:0/1413204764 conn(0x55e2bd476c70 msgr2=0x55e2bd477050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:29.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 -- 172.21.15.3:0/1413204764 shutdown_connections 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 -- 172.21.15.3:0/1413204764 wait complete. 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 -- start start 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.296+0000 7fdb2d208740 1 Processor -- start 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47a6e0 0x55e2bd482710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd482c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd483190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e2bd488310 con 0x55e2bd47a6e0 2024-08-01T04:25:29.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e2bd39acf0 con 0x55e2bd4858c0 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e2bd488490 con 0x55e2bd47b000 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb2d208740 10 client.0 _ll_register_callbacks cb 0x55e2bd386e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47a6e0 0x55e2bd482710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd482c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47a6e0 0x55e2bd482710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48180/0 (socket says 172.21.15.3:48180) 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb23e00640 1 -- 172.21.15.3:0/3785927617 learned_addr learned my addr 172.21.15.3:0/3785927617 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:29.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.297+0000 7fdb23e00640 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd483190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb22a00640 1 -- 172.21.15.3:0/3785927617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 msgr2=0x55e2bd483190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb22a00640 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd483190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb22a00640 1 -- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47a6e0 msgr2=0x55e2bd482710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb22a00640 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47a6e0 0x55e2bd482710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb22a00640 1 -- 172.21.15.3:0/3785927617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bd1513c0 con 0x55e2bd47b000 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb22a00640 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd482c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb240790f0 tx=0x7fdb10012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363366e478836672 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb0d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:25:29.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb0d600640 1 -- 172.21.15.3:0/3785927617 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10004050 con 0x55e2bd47b000 2024-08-01T04:25:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1c036040 con 0x55e2bd47b000 2024-08-01T04:25:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.298+0000 7fdb0d600640 1 -- 172.21.15.3:0/3785927617 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fdb100041f0 con 0x55e2bd47b000 2024-08-01T04:25:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.299+0000 7fdb0d600640 1 -- 172.21.15.3:0/3785927617 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb100043f0 con 0x55e2bd47b000 2024-08-01T04:25:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.299+0000 7fdb2d208740 10 client.26284 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.299+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2bd494090 con 0x55e2bd47b000 2024-08-01T04:25:29.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.299+0000 7fdb0d600640 1 -- 172.21.15.3:0/3785927617 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c036040 con 0x55e2bd47b000 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb0d600640 1 -- 172.21.15.3:0/3785927617 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdb10003a60 con 0x55e2bd47b000 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb2d208740 10 client.26284 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb2d208740 10 client.26284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb2d208740 20 client.26284 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb2d208740 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd4954a0 0x55e2bd4978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb2d208740 4 client.26284 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:25:29.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2bd3ed160 con 0x55e2bd4954a0 2024-08-01T04:25:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.300+0000 7fdb23e00640 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd4954a0 0x55e2bd4978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.301+0000 7fdb23e00640 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd4954a0 0x55e2bd4978b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2da644e9478828d server_cookie=df1fac67346441c7 in_seq=0 out_seq=0 2024-08-01T04:25:29.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.301+0000 7fdb0d600640 10 client.26284 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:25:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.301+0000 7fdb0d600640 1 -- 172.21.15.3:0/3785927617 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2bd3ed160 con 0x55e2bd4954a0 2024-08-01T04:25:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.301+0000 7fdb0d600640 10 client.26284 handle_command_reply: tid=0 2024-08-01T04:25:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.301+0000 7fdb2d208740 1 client.26284 shutdown 2024-08-01T04:25:29.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 msgr2=0x55e2bd482c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd482c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb240790f0 tx=0x7fdb10012a80 comp rx=0 tx=0).stop 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 shutdown_connections 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd4954a0 0x55e2bd4978b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd483190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd482c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 --2- 172.21.15.3:0/3785927617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47a6e0 0x55e2bd482710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb0d600640 0 client.26284 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:25:29.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 >> 172.21.15.3:0/3785927617 conn(0x55e2bd476c70 msgr2=0x55e2bd47c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:29.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 shutdown_connections 2024-08-01T04:25:29.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 1 -- 172.21.15.3:0/3785927617 wait complete. 2024-08-01T04:25:29.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.302+0000 7fdb2d208740 20 client.26284 trim_cache size 0 max 16384 2024-08-01T04:25:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 Processor -- start 2024-08-01T04:25:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 -- start start 2024-08-01T04:25:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd471910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd49d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04011220 con 0x55e2bd4858c0 2024-08-01T04:25:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400eaf0 con 0x55e2bd47a6e0 2024-08-01T04:25:29.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.317+0000 7fdb2d208740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400c960 con 0x55e2bd47b000 2024-08-01T04:25:29.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:33986/0 (socket says 172.21.15.3:33986) 2024-08-01T04:25:29.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 -- 172.21.15.3:0/4194519518 learned_addr learned my addr 172.21.15.3:0/4194519518 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:29.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb22a00640 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd471910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd49d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 -- 172.21.15.3:0/4194519518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 msgr2=0x55e2bd471910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd471910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 -- 172.21.15.3:0/4194519518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 msgr2=0x55e2bd49d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd49d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 -- 172.21.15.3:0/4194519518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bd4816c0 con 0x55e2bd47a6e0 2024-08-01T04:25:29.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.318+0000 7fdb23e00640 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47d140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0041e0 tx=0x7fdb1c004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6433044d11f45448 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:29.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.319+0000 7fdb0fe00640 1 -- 172.21.15.3:0/4194519518 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c003970 con 0x55e2bd47a6e0 2024-08-01T04:25:29.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.319+0000 7fdb0fe00640 1 -- 172.21.15.3:0/4194519518 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c036040 con 0x55e2bd47a6e0 2024-08-01T04:25:29.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.319+0000 7fdb0fe00640 1 -- 172.21.15.3:0/4194519518 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0317d0 con 0x55e2bd47a6e0 2024-08-01T04:25:29.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 -- 172.21.15.3:0/4194519518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 msgr2=0x55e2bd47d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47d140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0041e0 tx=0x7fdb1c004170 comp rx=0 tx=0).stop 2024-08-01T04:25:29.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 -- 172.21.15.3:0/4194519518 shutdown_connections 2024-08-01T04:25:29.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd4858c0 0x55e2bd49d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47b000 0x55e2bd471910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 --2- 172.21.15.3:0/4194519518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd47a6e0 0x55e2bd47d140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 -- 172.21.15.3:0/4194519518 >> 172.21.15.3:0/4194519518 conn(0x55e2bd476c70 msgr2=0x55e2bd484790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:29.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 -- 172.21.15.3:0/4194519518 shutdown_connections 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 -- 172.21.15.3:0/4194519518 wait complete. 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 -- start start 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.320+0000 7fdb2d208740 1 Processor -- start 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47a6e0 0x55e2bd48e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 0x55e2bd47f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd47fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400cfb0 con 0x55e2bd47b000 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400e420 con 0x55e2bd4858c0 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0400d680 con 0x55e2bd47a6e0 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb2d208740 10 client.0 _ll_register_callbacks cb 0x7fdb10009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 0x55e2bd47f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb22a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 0x55e2bd47f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48200/0 (socket says 172.21.15.3:48200) 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb23e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd47fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb22a00640 1 -- 172.21.15.3:0/1636512083 learned_addr learned my addr 172.21.15.3:0/1636512083 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:29.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.321+0000 7fdb23e00640 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47a6e0 0x55e2bd48e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb22a00640 1 -- 172.21.15.3:0/1636512083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47a6e0 msgr2=0x55e2bd48e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb22a00640 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47a6e0 0x55e2bd48e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb22a00640 1 -- 172.21.15.3:0/1636512083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 msgr2=0x55e2bd47fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb22a00640 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd47fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb22a00640 1 -- 172.21.15.3:0/1636512083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bd4ff190 con 0x55e2bd47b000 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb22a00640 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 0x55e2bd47f600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55e2bd481fa0 tx=0x7fdb10010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890213472a793f9a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb0d600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:29.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb0d600640 1 -- 172.21.15.3:0/1636512083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10010430 con 0x55e2bd47b000 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb0d600640 1 -- 172.21.15.3:0/1636512083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fdb100035f0 con 0x55e2bd47b000 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.322+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1c036040 con 0x55e2bd47b000 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb0d600640 1 -- 172.21.15.3:0/1636512083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb100105f0 con 0x55e2bd47b000 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 10 client.16128 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2bd5003a0 con 0x55e2bd47b000 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb0d600640 1 -- 172.21.15.3:0/1636512083 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c036040 con 0x55e2bd47b000 2024-08-01T04:25:29.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb0d600640 1 -- 172.21.15.3:0/1636512083 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fdb1000f640 con 0x55e2bd47b000 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 10 client.16128 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 10 client.16128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 20 client.16128 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd500b50 0x55e2bd502f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 4 client.16128 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.323+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2bd3e7670 con 0x55e2bd500b50 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.324+0000 7fdb23e00640 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd500b50 0x55e2bd502f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:29.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.324+0000 7fdb23e00640 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd500b50 0x55e2bd502f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27abde0c6a4f411b server_cookie=3d97be06741b459 in_seq=0 out_seq=0 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.324+0000 7fdb0d600640 10 client.16128 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.324+0000 7fdb0d600640 1 -- 172.21.15.3:0/1636512083 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2bd3e7670 con 0x55e2bd500b50 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.324+0000 7fdb0d600640 10 client.16128 handle_command_reply: tid=0 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.324+0000 7fdb2d208740 1 client.16128 shutdown 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 msgr2=0x55e2bd47f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 0x55e2bd47f600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55e2bd481fa0 tx=0x7fdb10010a80 comp rx=0 tx=0).stop 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 shutdown_connections 2024-08-01T04:25:29.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x55e2bd500b50 0x55e2bd502f70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2bd4858c0 0x55e2bd47fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e2bd47b000 0x55e2bd47f600 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 --2- 172.21.15.3:0/1636512083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e2bd47a6e0 0x55e2bd48e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb0d600640 0 client.16128 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 >> 172.21.15.3:0/1636512083 conn(0x55e2bd476c70 msgr2=0x55e2bd475480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 shutdown_connections 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 1 -- 172.21.15.3:0/1636512083 wait complete. 2024-08-01T04:25:29.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.325+0000 7fdb2d208740 20 client.16128 trim_cache size 0 max 16384 2024-08-01T04:25:29.336 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:25:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fdb080a2d70 msgr2=0x7fdb080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fdb080a2d70 0x7fdb080a5230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55e2bd477ad0 tx=0x7fdb140d2dc0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076450 msgr2=0x7fdb2419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076450 0x7fdb2419e6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e9f0 tx=0x7fdb1400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 shutdown_connections 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb241125c0 0x7fdb241a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fdb080a2d70 0x7fdb080a5230 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb24076d70 0x7fdb2419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 --2- 172.21.15.3:0/2817313099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb24076450 0x7fdb2419e6a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 >> 172.21.15.3:0/2817313099 conn(0x7fdb240ff5e0 msgr2=0x7fdb241019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 shutdown_connections 2024-08-01T04:25:29.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:29.326+0000 7fdb29000640 1 -- 172.21.15.3:0/2817313099 wait complete. 2024-08-01T04:25:29.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:29 smithi003 ceph-mon[32262]: pgmap v285: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:29.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:29 smithi160 ceph-mon[37224]: pgmap v285: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:30.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:29 smithi163 ceph-mon[37383]: pgmap v285: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:30.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:25:30.751 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:31.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 -- 172.21.15.3:0/3739694305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 msgr2=0x7f96e8103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 --2- 172.21.15.3:0/3739694305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e8103d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009930 tx=0x7f96d402a420 comp rx=0 tx=0).stop 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 -- 172.21.15.3:0/3739694305 shutdown_connections 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 --2- 172.21.15.3:0/3739694305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e8104280 0x7f96e81107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 --2- 172.21.15.3:0/3739694305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e8103d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 --2- 172.21.15.3:0/3739694305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e81098e0 0x7f96e8109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 -- 172.21.15.3:0/3739694305 >> 172.21.15.3:0/3739694305 conn(0x7f96e80ff5a0 msgr2=0x7f96e81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:31.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 -- 172.21.15.3:0/3739694305 shutdown_connections 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.553+0000 7f96ede00640 1 -- 172.21.15.3:0/3739694305 wait complete. 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.554+0000 7f96ede00640 1 Processor -- start 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.554+0000 7f96ede00640 1 -- start start 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ede00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ede00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e8104280 0x7f96e819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ede00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e81098e0 0x7f96e819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ede00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8113d30 con 0x7f96e8104280 2024-08-01T04:25:31.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ede00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8113bb0 con 0x7f96e81098e0 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ede00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8113eb0 con 0x7f96e81038e0 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e8104280 0x7f96e819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96eca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96eca00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:44990/0 (socket says 172.21.15.3:44990) 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96eca00640 1 -- 172.21.15.3:0/2519410630 learned_addr learned my addr 172.21.15.3:0/2519410630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.555+0000 7f96ed400640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e81098e0 0x7f96e819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96eca00640 1 -- 172.21.15.3:0/2519410630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e81098e0 msgr2=0x7f96e819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96eca00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e81098e0 0x7f96e819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96eca00640 1 -- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e8104280 msgr2=0x7f96e819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96eca00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e8104280 0x7f96e819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96eca00640 1 -- 172.21.15.3:0/2519410630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e819f180 con 0x7f96e81038e0 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96eca00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e819a1d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00e9f0 tx=0x7f96dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96e5600640 1 -- 172.21.15.3:0/2519410630 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00cd80 con 0x7f96e81038e0 2024-08-01T04:25:31.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96e5600640 1 -- 172.21.15.3:0/2519410630 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96dc010910 con 0x7f96e81038e0 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96e5600640 1 -- 172.21.15.3:0/2519410630 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc018990 con 0x7f96e81038e0 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.556+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e819f470 con 0x7f96e81038e0 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.557+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a6d50 con 0x7f96e81038e0 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.560+0000 7f96f1dce740 1 Processor -- start 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.560+0000 7f96e5600640 1 -- 172.21.15.3:0/2519410630 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f96dc010ac0 con 0x7f96e81038e0 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.560+0000 7f96f1dce740 1 -- start start 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e900ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e90b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96f1dce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c21e865bf0 con 0x55c21e9006e0 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96f1dce740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c21e7fdb20 con 0x55c21e901000 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96f1dce740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c21e861bb0 con 0x55c21e90b8c0 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e900ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e900ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48214/0 (socket says 172.21.15.3:48214) 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96ed400640 1 -- 172.21.15.3:0/2739907315 learned_addr learned my addr 172.21.15.3:0/2739907315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:31.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96eca00640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e90b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96e7e00640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96ed400640 1 -- 172.21.15.3:0/2739907315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 msgr2=0x55c21e90dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96ed400640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.561+0000 7f96ed400640 1 -- 172.21.15.3:0/2739907315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 msgr2=0x55c21e90b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96ed400640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e90b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96ed400640 1 -- 172.21.15.3:0/2739907315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21e8047f0 con 0x55c21e9006e0 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96eca00640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e90b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:25:31.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96e7e00640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96ed400640 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e900ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009fd0 tx=0x7f96e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3312c2e95de601 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96e5600640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f96d00a2cf0 0x7f96d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96d9c00640 1 -- 172.21.15.3:0/2739907315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0004d20 con 0x55c21e9006e0 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96d9c00640 1 -- 172.21.15.3:0/2739907315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0004050 con 0x55c21e9006e0 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96e5600640 1 -- 172.21.15.3:0/2519410630 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f96dc0cf7b0 con 0x7f96e81038e0 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.562+0000 7f96d9c00640 1 -- 172.21.15.3:0/2739907315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0005220 con 0x55c21e9006e0 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96eca00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f96d00a2cf0 0x7f96d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 -- 172.21.15.3:0/2739907315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 msgr2=0x55c21e900ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e900ac0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009fd0 tx=0x7f96e00118d0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 -- 172.21.15.3:0/2739907315 shutdown_connections 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e90b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2739907315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e900ac0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.563+0000 7f96f1dce740 1 -- 172.21.15.3:0/2739907315 >> 172.21.15.3:0/2739907315 conn(0x55c21e8fcc70 msgr2=0x55c21e8fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.564+0000 7f96f1dce740 1 -- 172.21.15.3:0/2739907315 shutdown_connections 2024-08-01T04:25:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.564+0000 7f96eca00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f96d00a2cf0 0x7f96d00a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55c21e8fdad0 tx=0x7f96dc000bc0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.564+0000 7f96f1dce740 1 -- 172.21.15.3:0/2739907315 wait complete. 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.564+0000 7f96f1dce740 1 -- start start 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.564+0000 7f96f1dce740 1 Processor -- start 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e9006e0 0x55c21e908760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 0x55c21e908ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e9091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c21e90e4e0 con 0x55c21e901000 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c21e820cf0 con 0x55c21e9006e0 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c21e90e660 con 0x55c21e90b8c0 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96f1dce740 10 client.0 _ll_register_callbacks cb 0x55c21e80ce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 0x55c21e908ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96e7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 0x55c21e908ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48222/0 (socket says 172.21.15.3:48222) 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96e7e00640 1 -- 172.21.15.3:0/431694823 learned_addr learned my addr 172.21.15.3:0/431694823 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96ed400640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e9006e0 0x55c21e908760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.565+0000 7f96ed400640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e9091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96e7e00640 1 -- 172.21.15.3:0/431694823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 msgr2=0x55c21e9091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96e7e00640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e9091e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96e7e00640 1 -- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e9006e0 msgr2=0x55c21e908760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96e7e00640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e9006e0 0x55c21e908760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96e7e00640 1 -- 172.21.15.3:0/431694823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21e5d73c0 con 0x55c21e901000 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96e7e00640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 0x55c21e908ca0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96d4033000 tx=0x7f96d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b061e24b931ace70 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.566+0000 7f96ce000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.567+0000 7f96ce000640 1 -- 172.21.15.3:0/431694823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4035040 con 0x55c21e901000 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.567+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e0004050 con 0x55c21e901000 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.567+0000 7f96ce000640 1 -- 172.21.15.3:0/431694823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f96d4005230 con 0x55c21e901000 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.567+0000 7f96ce000640 1 -- 172.21.15.3:0/431694823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96d4005430 con 0x55c21e901000 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.567+0000 7f96f1dce740 10 client.16140 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.567+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c21e91a140 con 0x55c21e901000 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96ce000640 1 -- 172.21.15.3:0/431694823 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0004050 con 0x55c21e901000 2024-08-01T04:25:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96ce000640 1 -- 172.21.15.3:0/431694823 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f96d4031880 con 0x55c21e901000 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96f1dce740 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96f1dce740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96f1dce740 20 client.16140 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96f1dce740 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e91b600 0x55c21e91d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96f1dce740 4 client.16140 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c21e873160 con 0x55c21e91b600 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.568+0000 7f96ed400640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e91b600 0x55c21e91d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.569+0000 7f96ed400640 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e91b600 0x55c21e91d9f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8342df032975520d server_cookie=d6733560f1a17d80 in_seq=0 out_seq=0 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.569+0000 7f96ce000640 10 client.16140 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96ce000640 1 -- 172.21.15.3:0/431694823 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c21e873160 con 0x55c21e91b600 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96ce000640 10 client.16140 handle_command_reply: tid=0 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 client.16140 shutdown 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 msgr2=0x55c21e908ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 0x55c21e908ca0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96d4033000 tx=0x7f96d4004270 comp rx=0 tx=0).stop 2024-08-01T04:25:31.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 shutdown_connections 2024-08-01T04:25:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e91b600 0x55c21e91d9f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e9091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e901000 0x55c21e908ca0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 --2- 172.21.15.3:0/431694823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e9006e0 0x55c21e908760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 >> 172.21.15.3:0/431694823 conn(0x55c21e8fcc70 msgr2=0x55c21e9029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.570+0000 7f96ce000640 0 client.16140 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.571+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 shutdown_connections 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.571+0000 7f96f1dce740 1 -- 172.21.15.3:0/431694823 wait complete. 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.571+0000 7f96f1dce740 20 client.16140 trim_cache size 0 max 16384 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 Processor -- start 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 -- start start 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e903140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e9232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e923800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81b2190 con 0x55c21e9006e0 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81afa60 con 0x55c21e901000 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.586+0000 7f96f1dce740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81ad8d0 con 0x55c21e90b8c0 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e903140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e903140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48230/0 (socket says 172.21.15.3:48230) 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 -- 172.21.15.3:0/2195576426 learned_addr learned my addr 172.21.15.3:0/2195576426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96e7e00640 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e9232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e923800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 -- 172.21.15.3:0/2195576426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 msgr2=0x55c21e923800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e923800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 -- 172.21.15.3:0/2195576426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 msgr2=0x55c21e9232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e9232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.587+0000 7f96ed400640 1 -- 172.21.15.3:0/2195576426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21e90b2b0 con 0x55c21e9006e0 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.588+0000 7f96ed400640 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e903140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96e0006010 tx=0x7f96e0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0e6e5694a482af server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.588+0000 7f96d9200640 1 -- 172.21.15.3:0/2195576426 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0004050 con 0x55c21e9006e0 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.588+0000 7f96d9200640 1 -- 172.21.15.3:0/2195576426 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e00046d0 con 0x55c21e9006e0 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.588+0000 7f96d9200640 1 -- 172.21.15.3:0/2195576426 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e000fcb0 con 0x55c21e9006e0 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 -- 172.21.15.3:0/2195576426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 msgr2=0x55c21e903140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e903140 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96e0006010 tx=0x7f96e0009be0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 -- 172.21.15.3:0/2195576426 shutdown_connections 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e923800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e9232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 --2- 172.21.15.3:0/2195576426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e903140 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 -- 172.21.15.3:0/2195576426 >> 172.21.15.3:0/2195576426 conn(0x55c21e8fcc70 msgr2=0x55c21e903a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 -- 172.21.15.3:0/2195576426 shutdown_connections 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 -- 172.21.15.3:0/2195576426 wait complete. 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 -- start start 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.589+0000 7f96f1dce740 1 Processor -- start 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e913bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e914100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81adf20 con 0x55c21e9006e0 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81af390 con 0x55c21e901000 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81ae5f0 con 0x55c21e90b8c0 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96f1dce740 10 client.0 _ll_register_callbacks cb 0x7f96d4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e913bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96ed400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e913bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48242/0 (socket says 172.21.15.3:48242) 2024-08-01T04:25:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96ed400640 1 -- 172.21.15.3:0/462774803 learned_addr learned my addr 172.21.15.3:0/462774803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.590+0000 7f96e7e00640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e914100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 -- 172.21.15.3:0/462774803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 msgr2=0x55c21e90ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 -- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 msgr2=0x55c21e914100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e914100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 -- 172.21.15.3:0/462774803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21e90f7b0 con 0x55c21e9006e0 2024-08-01T04:25:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96e7e00640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e914100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ed400640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e913bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f96e000f990 tx=0x7f96e000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1b2c191d6de475 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ce000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ce000640 1 -- 172.21.15.3:0/462774803 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e000fa30 con 0x55c21e9006e0 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e00046d0 con 0x55c21e9006e0 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.591+0000 7f96ce000640 1 -- 172.21.15.3:0/462774803 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f96e000fbd0 con 0x55c21e9006e0 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96ce000640 1 -- 172.21.15.3:0/462774803 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96e000fdd0 con 0x55c21e9006e0 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:31.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c21e98e660 con 0x55c21e9006e0 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96ce000640 1 -- 172.21.15.3:0/462774803 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e00046d0 con 0x55c21e9006e0 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96ce000640 1 -- 172.21.15.3:0/462774803 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f96e0025040 con 0x55c21e9006e0 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 20 client.16146 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.592+0000 7f96f1dce740 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e98eb80 0x55c21e990f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.593+0000 7f96f1dce740 4 client.16146 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:25:31.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.593+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c21e86d670 con 0x55c21e98eb80 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.593+0000 7f96e7e00640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e98eb80 0x55c21e990f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.593+0000 7f96e7e00640 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e98eb80 0x55c21e990f70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6772ee95b78266a server_cookie=bd7cedcab3ce6d3d in_seq=0 out_seq=0 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.593+0000 7f96ce000640 10 client.16146 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96ce000640 1 -- 172.21.15.3:0/462774803 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c21e86d670 con 0x55c21e98eb80 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96ce000640 10 client.16146 handle_command_reply: tid=0 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 client.16146 shutdown 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 msgr2=0x55c21e913bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e913bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f96e000f990 tx=0x7f96e000f890 comp rx=0 tx=0).stop 2024-08-01T04:25:31.653 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:25:31.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 shutdown_connections 2024-08-01T04:25:31.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c21e98eb80 0x55c21e990f70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c21e90b8c0 0x55c21e90ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c21e901000 0x55c21e914100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 --2- 172.21.15.3:0/462774803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c21e9006e0 0x55c21e913bc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 >> 172.21.15.3:0/462774803 conn(0x55c21e8fcc70 msgr2=0x55c21e909fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:31.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96ce000640 0 client.16146 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:31.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 shutdown_connections 2024-08-01T04:25:31.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 1 -- 172.21.15.3:0/462774803 wait complete. 2024-08-01T04:25:31.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.594+0000 7f96f1dce740 20 client.16146 trim_cache size 0 max 16384 2024-08-01T04:25:31.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f96d00a2cf0 msgr2=0x7f96d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f96d00a2cf0 0x7f96d00a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55c21e8fdad0 tx=0x7f96dc000bc0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 msgr2=0x7f96e819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:31.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e819a1d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00e9f0 tx=0x7f96dc00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 shutdown_connections 2024-08-01T04:25:31.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e81098e0 0x7f96e819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96e8104280 0x7f96e819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f96d00a2cf0 0x7f96d00a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 --2- 172.21.15.3:0/2519410630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96e81038e0 0x7f96e819a1d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:31.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 >> 172.21.15.3:0/2519410630 conn(0x7f96e80ff5a0 msgr2=0x7f96e80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:31.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 shutdown_connections 2024-08-01T04:25:31.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:31.595+0000 7f96ede00640 1 -- 172.21.15.3:0/2519410630 wait complete. 2024-08-01T04:25:31.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:31 smithi160 ceph-mon[37224]: pgmap v286: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-08-01T04:25:32.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:31 smithi163 ceph-mon[37383]: pgmap v286: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-08-01T04:25:32.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:31 smithi003 ceph-mon[32262]: pgmap v286: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-08-01T04:25:33.594 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:25:33.594 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:25:33.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:33 smithi160 ceph-mon[37224]: pgmap v287: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:34.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:33 smithi163 ceph-mon[37383]: pgmap v287: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:34.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:33 smithi003 ceph-mon[32262]: pgmap v287: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:34.080 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 -- 172.21.15.3:0/1344762166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076450 msgr2=0x7f8130076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 --2- 172.21.15.3:0/1344762166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076450 0x7f8130076830 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8128009900 tx=0x7f812802a3f0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 -- 172.21.15.3:0/1344762166 shutdown_connections 2024-08-01T04:25:34.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 --2- 172.21.15.3:0/1344762166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81301125a0 0x7f8130114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 --2- 172.21.15.3:0/1344762166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076d70 0x7f8130112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 --2- 172.21.15.3:0/1344762166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076450 0x7f8130076830 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.731+0000 7f8135200640 1 -- 172.21.15.3:0/1344762166 >> 172.21.15.3:0/1344762166 conn(0x7f81300ff640 msgr2=0x7f8130101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.732+0000 7f8135200640 1 -- 172.21.15.3:0/1344762166 shutdown_connections 2024-08-01T04:25:34.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.732+0000 7f8135200640 1 -- 172.21.15.3:0/1344762166 wait complete. 2024-08-01T04:25:34.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 Processor -- start 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 -- start start 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 0x7f813019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076d70 0x7f813019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81301125a0 0x7f81301a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8130118120 con 0x7f8130076d70 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8130117fa0 con 0x7f8130076450 2024-08-01T04:25:34.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.733+0000 7f8135200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81301182a0 con 0x7f81301125a0 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 0x7f813019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 0x7f813019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45428/0 (socket says 172.21.15.3:45428) 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 -- 172.21.15.3:0/4113876138 learned_addr learned my addr 172.21.15.3:0/4113876138 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812ea00640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076d70 0x7f813019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812fe00640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81301125a0 0x7f81301a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 -- 172.21.15.3:0/4113876138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81301125a0 msgr2=0x7f81301a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81301125a0 0x7f81301a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 -- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076d70 msgr2=0x7f813019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076d70 0x7f813019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812f400640 1 -- 172.21.15.3:0/4113876138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81301a3590 con 0x7f8130076450 2024-08-01T04:25:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.734+0000 7f812ea00640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076d70 0x7f813019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:34.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.735+0000 7f812f400640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 0x7f813019e5e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8128006010 tx=0x7f8128004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:34.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.735+0000 7f8127e00640 1 -- 172.21.15.3:0/4113876138 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812803f030 con 0x7f8130076450 2024-08-01T04:25:34.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.736+0000 7f8127e00640 1 -- 172.21.15.3:0/4113876138 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8128004440 con 0x7f8130076450 2024-08-01T04:25:34.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.736+0000 7f8127e00640 1 -- 172.21.15.3:0/4113876138 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128035070 con 0x7f8130076450 2024-08-01T04:25:34.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.736+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a3820 con 0x7f8130076450 2024-08-01T04:25:34.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.736+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a3cb0 con 0x7f8130076450 2024-08-01T04:25:34.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.739+0000 7f81393ba740 1 Processor -- start 2024-08-01T04:25:34.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.739+0000 7f81393ba740 1 -- start start 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.739+0000 7f8127e00640 1 -- 172.21.15.3:0/4113876138 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f8128042030 con 0x7f8130076450 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f81393ba740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a56a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f81393ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f81393ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a63c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f81393ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c5769717e0 con 0x55c576a61830 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f81393ba740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c5769bcac0 con 0x55c576a56f70 2024-08-01T04:25:34.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f81393ba740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c5769548c0 con 0x55c576a56650 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f812f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f812fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a56a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f812ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a63c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f812f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45440/0 (socket says 172.21.15.3:45440) 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.740+0000 7f812f400640 1 -- 172.21.15.3:0/917434685 learned_addr learned my addr 172.21.15.3:0/917434685 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f812f400640 1 -- 172.21.15.3:0/917434685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 msgr2=0x55c576a56a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f812f400640 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a56a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f812f400640 1 -- 172.21.15.3:0/917434685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 msgr2=0x55c576a63c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f812f400640 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a63c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f812f400640 1 -- 172.21.15.3:0/917434685 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57695b760 con 0x55c576a56f70 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f812f400640 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a612f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81280eca90 tx=0x7f81280f56d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8982710eff91b3ee server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f8124c00640 1 -- 172.21.15.3:0/917434685 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81280efcd0 con 0x55c576a56f70 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f8124c00640 1 -- 172.21.15.3:0/917434685 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81280fc030 con 0x55c576a56f70 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.741+0000 7f8124c00640 1 -- 172.21.15.3:0/917434685 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81281019f0 con 0x55c576a56f70 2024-08-01T04:25:34.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f8127e00640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f81140a2d70 0x7f81140a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f8127e00640 1 -- 172.21.15.3:0/4113876138 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f81280ef4d0 con 0x7f8130076450 2024-08-01T04:25:34.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f812fe00640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f81140a2d70 0x7f81140a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 -- 172.21.15.3:0/917434685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 msgr2=0x55c576a612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a612f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81280eca90 tx=0x7f81280f56d0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 -- 172.21.15.3:0/917434685 shutdown_connections 2024-08-01T04:25:34.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a63c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a612f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 --2- 172.21.15.3:0/917434685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a56a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.742+0000 7f81393ba740 1 -- 172.21.15.3:0/917434685 >> 172.21.15.3:0/917434685 conn(0x55c576a52be0 msgr2=0x55c576a52fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.743+0000 7f81393ba740 1 -- 172.21.15.3:0/917434685 shutdown_connections 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.743+0000 7f81393ba740 1 -- 172.21.15.3:0/917434685 wait complete. 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.743+0000 7f812fe00640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f81140a2d70 0x7f81140a5230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f813006a420 tx=0x7f8120005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.743+0000 7f81393ba740 1 -- start start 2024-08-01T04:25:34.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 Processor -- start 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a5e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a5f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c576977ae0 con 0x55c576a61830 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55c5769b8a80 con 0x55c576a56f70 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55c576a641c0 con 0x55c576a56650 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f81393ba740 10 client.0 _ll_register_callbacks cb 0x55c576979190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:34.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f812ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f812f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a5e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f812ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a5eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45456/0 (socket says 172.21.15.3:45456) 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f812ea00640 1 -- 172.21.15.3:0/1042620181 learned_addr learned my addr 172.21.15.3:0/1042620181 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.744+0000 7f812fe00640 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a5f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f812f400640 1 -- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 msgr2=0x55c576a5eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f812f400640 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a5eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f812f400640 1 -- 172.21.15.3:0/1042620181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 msgr2=0x55c576a5f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f812f400640 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a5f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f812f400640 1 -- 172.21.15.3:0/1042620181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57672e6a0 con 0x55c576a56650 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f812f400640 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a5e660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81280b7e30 tx=0x7f81280f16c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbcfd4fe4448ba0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f811d600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.745+0000 7f811d600640 1 -- 172.21.15.3:0/1042620181 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128049070 con 0x55c576a56650 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81280fc030 con 0x55c576a56650 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f811d600640 1 -- 172.21.15.3:0/1042620181 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f81280f5160 con 0x55c576a56650 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f811d600640 1 -- 172.21.15.3:0/1042620181 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f81280b7c70 con 0x55c576a56650 2024-08-01T04:25:34.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 10 client.26320 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c576a6fc70 con 0x55c576a56650 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f811d600640 1 -- 172.21.15.3:0/1042620181 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81280fc030 con 0x55c576a56650 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f811d600640 1 -- 172.21.15.3:0/1042620181 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f812804c060 con 0x55c576a56650 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 10 client.26320 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 20 client.26320 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a70fc0 0x55c576a733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 4 client.26320 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:25:34.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.746+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5769ca030 con 0x55c576a70fc0 2024-08-01T04:25:34.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.747+0000 7f812ea00640 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a70fc0 0x55c576a733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.748+0000 7f812ea00640 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a70fc0 0x55c576a733b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4147c8401a39ec0f server_cookie=3cd3434eb083a387 in_seq=0 out_seq=0 2024-08-01T04:25:34.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.748+0000 7f811d600640 10 client.26320 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:34.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f811d600640 1 -- 172.21.15.3:0/1042620181 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c5769ca030 con 0x55c576a70fc0 2024-08-01T04:25:34.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f811d600640 10 client.26320 handle_command_reply: tid=0 2024-08-01T04:25:34.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 client.26320 shutdown 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 msgr2=0x55c576a5e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a5e660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81280b7e30 tx=0x7f81280f16c0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 shutdown_connections 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a70fc0 0x55c576a733b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a61830 0x55c576a5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a56f70 0x55c576a5eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 --2- 172.21.15.3:0/1042620181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56650 0x55c576a5e660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f811d600640 0 client.26320 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:34.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 >> 172.21.15.3:0/1042620181 conn(0x55c576a52be0 msgr2=0x55c576a588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:34.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.749+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 shutdown_connections 2024-08-01T04:25:34.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.750+0000 7f81393ba740 1 -- 172.21.15.3:0/1042620181 wait complete. 2024-08-01T04:25:34.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.750+0000 7f81393ba740 20 client.26320 trim_cache size 0 max 16384 2024-08-01T04:25:34.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.769+0000 7f81393ba740 1 Processor -- start 2024-08-01T04:25:34.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.769+0000 7f81393ba740 1 -- start start 2024-08-01T04:25:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f81393ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a51c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f81393ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a78b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f81393ba740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a79060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f81393ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b65c0 con 0x55c576a56650 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f81393ba740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b3e90 con 0x55c576a52be0 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f81393ba740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b1d00 con 0x55c576a56f70 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a78b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a78b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55996/0 (socket says 172.21.15.3:55996) 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 -- 172.21.15.3:0/3178040257 learned_addr learned my addr 172.21.15.3:0/3178040257 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812f400640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a51c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812fe00640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a79060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 -- 172.21.15.3:0/3178040257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 msgr2=0x55c576a79060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a79060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 -- 172.21.15.3:0/3178040257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 msgr2=0x55c576a51c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a51c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812ea00640 1 -- 172.21.15.3:0/3178040257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c576a52a10 con 0x55c576a56650 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812fe00640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a79060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-01T04:25:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.770+0000 7f812f400640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a51c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:34.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.771+0000 7f812ea00640 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a78b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81180098d0 tx=0x7f8118009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869e0d894a9f5a3a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.771+0000 7f811fe00640 1 -- 172.21.15.3:0/3178040257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180043c0 con 0x55c576a56650 2024-08-01T04:25:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.771+0000 7f811fe00640 1 -- 172.21.15.3:0/3178040257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118004560 con 0x55c576a56650 2024-08-01T04:25:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.771+0000 7f811fe00640 1 -- 172.21.15.3:0/3178040257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811801e610 con 0x55c576a56650 2024-08-01T04:25:34.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 -- 172.21.15.3:0/3178040257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 msgr2=0x55c576a78b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a78b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81180098d0 tx=0x7f8118009500 comp rx=0 tx=0).stop 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 -- 172.21.15.3:0/3178040257 shutdown_connections 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a79060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a78b20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 --2- 172.21.15.3:0/3178040257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a51c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 -- 172.21.15.3:0/3178040257 >> 172.21.15.3:0/3178040257 conn(0x55c576a5a050 msgr2=0x55c576a51520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 -- 172.21.15.3:0/3178040257 shutdown_connections 2024-08-01T04:25:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.772+0000 7f81393ba740 1 -- 172.21.15.3:0/3178040257 wait complete. 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.773+0000 7f81393ba740 1 -- start start 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.773+0000 7f81393ba740 1 Processor -- start 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.773+0000 7f81393ba740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a69450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.773+0000 7f81393ba740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a69990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.773+0000 7f81393ba740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a64400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.773+0000 7f81393ba740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b2350 con 0x55c576a56650 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f81393ba740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b37c0 con 0x55c576a52be0 2024-08-01T04:25:34.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f81393ba740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f81301b2a20 con 0x55c576a56f70 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f81393ba740 10 client.0 _ll_register_callbacks cb 0x7f8120009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a69990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a69990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55998/0 (socket says 172.21.15.3:55998) 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 -- 172.21.15.3:0/155596442 learned_addr learned my addr 172.21.15.3:0/155596442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812f400640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a69450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812fe00640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a64400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 -- 172.21.15.3:0/155596442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 msgr2=0x55c576a64400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a64400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 -- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 msgr2=0x55c576a69450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a69450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.774+0000 7f812ea00640 1 -- 172.21.15.3:0/155596442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c576a64940 con 0x55c576a56650 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f812ea00640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a69990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8118005b40 tx=0x7f8118005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5cf44623dfc329 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f811d600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f811d600640 1 -- 172.21.15.3:0/155596442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118004c40 con 0x55c576a56650 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118004560 con 0x55c576a56650 2024-08-01T04:25:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f811d600640 1 -- 172.21.15.3:0/155596442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7f8118004de0 con 0x55c576a56650 2024-08-01T04:25:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f811d600640 1 -- 172.21.15.3:0/155596442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8118005050 con 0x55c576a56650 2024-08-01T04:25:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f81393ba740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.775+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c576adc330 con 0x55c576a56650 2024-08-01T04:25:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f811d600640 1 -- 172.21.15.3:0/155596442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118004560 con 0x55c576a56650 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f811d600640 1 -- 172.21.15.3:0/155596442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8118026070 con 0x55c576a56650 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f81393ba740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f81393ba740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f81393ba740 20 client.16164 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f81393ba740 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a61830 0x55c576adea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f81393ba740 4 client.16164 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c576a4de50 con 0x55c576a61830 2024-08-01T04:25:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.776+0000 7f812f400640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a61830 0x55c576adea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.777+0000 7f812f400640 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a61830 0x55c576adea10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f735fff217ae6dd server_cookie=cc73897338bef672 in_seq=0 out_seq=0 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.777+0000 7f811d600640 10 client.16164 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f811d600640 1 -- 172.21.15.3:0/155596442 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c576a4de50 con 0x55c576a61830 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f811d600640 10 client.16164 handle_command_reply: tid=0 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 client.16164 shutdown 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 msgr2=0x55c576a69990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a69990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8118005b40 tx=0x7f8118005fa0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 shutdown_connections 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55c576a61830 0x55c576adea10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c576a56f70 0x55c576a64400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c576a56650 0x55c576a69990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 --2- 172.21.15.3:0/155596442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c576a52be0 0x55c576a69450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 >> 172.21.15.3:0/155596442 conn(0x55c576a5a050 msgr2=0x55c576a5ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.778+0000 7f811d600640 0 client.16164 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.779+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 shutdown_connections 2024-08-01T04:25:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.779+0000 7f81393ba740 1 -- 172.21.15.3:0/155596442 wait complete. 2024-08-01T04:25:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.779+0000 7f81393ba740 20 client.16164 trim_cache size 0 max 16384 2024-08-01T04:25:34.787 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:25:34.787 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:25:34.787 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "171d234e-d449-4740-bf30-ed5e0e095c78", 2024-08-01T04:25:34.787 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:25:34.788 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:25:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f81140a2d70 msgr2=0x7f81140a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f81140a2d70 0x7f81140a5230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f813006a420 tx=0x7f8120005f90 comp rx=0 tx=0).stop 2024-08-01T04:25:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 msgr2=0x7f813019e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 0x7f813019e5e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8128006010 tx=0x7f8128004040 comp rx=0 tx=0).stop 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 shutdown_connections 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f81140a2d70 0x7f81140a5230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81301125a0 0x7f81301a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8130076d70 0x7f813019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 --2- 172.21.15.3:0/4113876138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8130076450 0x7f813019e5e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 >> 172.21.15.3:0/4113876138 conn(0x7f81300ff640 msgr2=0x7f8130101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 shutdown_connections 2024-08-01T04:25:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:34.780+0000 7f8135200640 1 -- 172.21.15.3:0/4113876138 wait complete. 2024-08-01T04:25:35.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: pgmap v288: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:25:35.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:25:35.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:25:35.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:25:35.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:25:35.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:35 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:25:35.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: pgmap v288: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:25:35.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:25:35.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:35 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: pgmap v288: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:25:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:25:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:25:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:25:36.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:25:36.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:35 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.376 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:25:36.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:36 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:25:36.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:36 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:36 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:25:36.899 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:36 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:36 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:25:36.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:36 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:36.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:36 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:25:36.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:36 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:36 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:25:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:36 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:36 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:25:37.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:36 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:25:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 -- 172.21.15.3:0/10874573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8780742f0 msgr2=0x7fa8780746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 --2- 172.21.15.3:0/10874573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8780742f0 0x7fa8780746d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa868009900 tx=0x7fa86802a480 comp rx=0 tx=0).stop 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 -- 172.21.15.3:0/10874573 shutdown_connections 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 --2- 172.21.15.3:0/10874573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 0x7fa878110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 --2- 172.21.15.3:0/10874573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa878074c10 0x7fa87810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 --2- 172.21.15.3:0/10874573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa8780742f0 0x7fa8780746d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 -- 172.21.15.3:0/10874573 >> 172.21.15.3:0/10874573 conn(0x7fa8780fd5e0 msgr2=0x7fa8780ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 -- 172.21.15.3:0/10874573 shutdown_connections 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.032+0000 7fa87e600640 1 -- 172.21.15.3:0/10874573 wait complete. 2024-08-01T04:25:37.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.034+0000 7fa87e600640 1 Processor -- start 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.034+0000 7fa87e600640 1 -- start start 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa87e600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 0x7fa87819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa87e600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878074c10 0x7fa87819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa87e600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 0x7fa87819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa87e600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa878113f00 con 0x7fa878074c10 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa87e600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa878113d80 con 0x7fa8780742f0 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa87e600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa878114080 con 0x7fa87810e160 2024-08-01T04:25:37.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa877400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 0x7fa87819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa877400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 0x7fa87819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45500/0 (socket says 172.21.15.3:45500) 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa877400640 1 -- 172.21.15.3:0/3702665041 learned_addr learned my addr 172.21.15.3:0/3702665041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa877e00640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 0x7fa87819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.035+0000 7fa876a00640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878074c10 0x7fa87819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877400640 1 -- 172.21.15.3:0/3702665041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 msgr2=0x7fa87819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877400640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 0x7fa87819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877400640 1 -- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878074c10 msgr2=0x7fa87819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877400640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878074c10 0x7fa87819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877400640 1 -- 172.21.15.3:0/3702665041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87819f390 con 0x7fa8780742f0 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877e00640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 0x7fa87819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa877400640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 0x7fa87819a3e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa868004770 tx=0x7fa868003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa86fe00640 1 -- 172.21.15.3:0/3702665041 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86803f070 con 0x7fa8780742f0 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87819f620 con 0x7fa8780742f0 2024-08-01T04:25:37.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.036+0000 7fa86fe00640 1 -- 172.21.15.3:0/3702665041 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa868031ca0 con 0x7fa8780742f0 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.037+0000 7fa86fe00640 1 -- 172.21.15.3:0/3702665041 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868035030 con 0x7fa8780742f0 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.037+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8781a6f60 con 0x7fa8780742f0 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.040+0000 7fa86fe00640 1 -- 172.21.15.3:0/3702665041 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa868042070 con 0x7fa8780742f0 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.040+0000 7fa880fe0740 1 Processor -- start 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.040+0000 7fa880fe0740 1 -- start start 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c03192b90 0x555c03192f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c0319d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 0x555c031a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa880fe0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c030f8bf0 con 0x555c03192b90 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa880fe0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x555c03090bd0 con 0x555c031934b0 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa880fe0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555c030f4bb0 con 0x555c0319dd70 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa877e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c03192b90 0x555c03192f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa877400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c0319d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa876a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 0x555c031a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa876a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 0x555c031a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43028/0 (socket says 172.21.15.3:43028) 2024-08-01T04:25:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa876a00640 1 -- 172.21.15.3:0/589266332 learned_addr learned my addr 172.21.15.3:0/589266332 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.041+0000 7fa877e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c03192b90 0x555c03192f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56030/0 (socket says 172.21.15.3:56030) 2024-08-01T04:25:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa876a00640 1 -- 172.21.15.3:0/589266332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 msgr2=0x555c0319d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa876a00640 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c0319d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa876a00640 1 -- 172.21.15.3:0/589266332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c03192b90 msgr2=0x555c03192f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa876a00640 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c03192b90 0x555c03192f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa876a00640 1 -- 172.21.15.3:0/589266332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c030978a0 con 0x555c0319dd70 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa86fe00640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa85c0a2d70 0x7fa85c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa876a00640 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 0x555c031a0160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa864009c90 tx=0x7fa864011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7bf9e4a2b8321b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.042+0000 7fa86fe00640 1 -- 172.21.15.3:0/3702665041 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fa8680efa30 con 0x7fa8780742f0 2024-08-01T04:25:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa877e00640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa85c0a2d70 0x7fa85c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa86cc00640 1 -- 172.21.15.3:0/589266332 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86401e070 con 0x555c0319dd70 2024-08-01T04:25:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa86cc00640 1 -- 172.21.15.3:0/589266332 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864019b10 con 0x555c0319dd70 2024-08-01T04:25:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa86cc00640 1 -- 172.21.15.3:0/589266332 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86400ea70 con 0x555c0319dd70 2024-08-01T04:25:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa880fe0740 1 -- 172.21.15.3:0/589266332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 msgr2=0x555c031a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa877e00640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa85c0a2d70 0x7fa85c0a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa870009fd0 tx=0x7fa870009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.043+0000 7fa880fe0740 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 0x555c031a0160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa864009c90 tx=0x7fa864011810 comp rx=0 tx=0).stop 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 -- 172.21.15.3:0/589266332 shutdown_connections 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c0319dd70 0x555c031a0160 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c0319d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 --2- 172.21.15.3:0/589266332 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c03192b90 0x555c03192f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 -- 172.21.15.3:0/589266332 >> 172.21.15.3:0/589266332 conn(0x555c0318f120 msgr2=0x555c0318f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 -- 172.21.15.3:0/589266332 shutdown_connections 2024-08-01T04:25:37.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 -- 172.21.15.3:0/589266332 wait complete. 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 -- start start 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.044+0000 7fa880fe0740 1 Processor -- start 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c03192b90 0x555c0319ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 0x555c0319b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c0319b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555c031a0940 con 0x555c0319dd70 2024-08-01T04:25:37.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x555c030b3d40 con 0x555c03192b90 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x555c031a0ac0 con 0x555c031934b0 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa880fe0740 10 client.0 _ll_register_callbacks cb 0x555c0307cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa876a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 0x555c0319b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa876a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 0x555c0319b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43036/0 (socket says 172.21.15.3:43036) 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa876a00640 1 -- 172.21.15.3:0/2439124697 learned_addr learned my addr 172.21.15.3:0/2439124697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa877e00640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c0319b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.045+0000 7fa877400640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c03192b90 0x555c0319ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa876a00640 1 -- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c03192b90 msgr2=0x555c0319ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa876a00640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c03192b90 0x555c0319ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa876a00640 1 -- 172.21.15.3:0/2439124697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 msgr2=0x555c0319b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa876a00640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c0319b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa876a00640 1 -- 172.21.15.3:0/2439124697 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c02e6a3c0 con 0x555c031934b0 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa877e00640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c0319b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa876a00640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 0x555c0319b150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa86401c040 tx=0x7fa8640199b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4211b5fc29773d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.046+0000 7fa861600640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa861600640 1 -- 172.21.15.3:0/2439124697 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86401e050 con 0x555c031934b0 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa864019b10 con 0x555c031934b0 2024-08-01T04:25:37.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa861600640 1 -- 172.21.15.3:0/2439124697 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fa864005450 con 0x555c031934b0 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa861600640 1 -- 172.21.15.3:0/2439124697 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa864005650 con 0x555c031934b0 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa880fe0740 10 client.26335 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c031ac5d0 con 0x555c031934b0 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.047+0000 7fa861600640 1 -- 172.21.15.3:0/2439124697 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa864019b10 con 0x555c031934b0 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa861600640 1 -- 172.21.15.3:0/2439124697 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa8640046c0 con 0x555c031934b0 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa880fe0740 10 client.26335 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:37.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa880fe0740 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa880fe0740 20 client.26335 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c031ad9e0 0x555c031afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa880fe0740 4 client.26335 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c03106160 con 0x555c031ad9e0 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.048+0000 7fa877400640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c031ad9e0 0x555c031afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.049+0000 7fa877400640 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c031ad9e0 0x555c031afdd0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e273c01130c1f255 server_cookie=cd902f1133c6b142 in_seq=0 out_seq=0 2024-08-01T04:25:37.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.049+0000 7fa861600640 10 client.26335 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa861600640 1 -- 172.21.15.3:0/2439124697 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555c03106160 con 0x555c031ad9e0 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa861600640 10 client.26335 handle_command_reply: tid=0 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 client.26335 shutdown 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 msgr2=0x555c0319b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 0x555c0319b150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa86401c040 tx=0x7fa8640199b0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 shutdown_connections 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c031ad9e0 0x555c031afdd0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c0319b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c031934b0 0x555c0319b150 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2439124697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c03192b90 0x555c0319ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa861600640 0 client.26335 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 >> 172.21.15.3:0/2439124697 conn(0x555c0318f120 msgr2=0x555c03194e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.050+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 shutdown_connections 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.051+0000 7fa880fe0740 1 -- 172.21.15.3:0/2439124697 wait complete. 2024-08-01T04:25:37.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.051+0000 7fa880fe0740 20 client.26335 trim_cache size 0 max 16384 2024-08-01T04:25:37.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 Processor -- start 2024-08-01T04:25:37.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 -- start start 2024-08-01T04:25:37.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c03194d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c031b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c031b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781b2330 con 0x555c0319dd70 2024-08-01T04:25:37.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781afc00 con 0x555c031934b0 2024-08-01T04:25:37.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.070+0000 7fa880fe0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ada70 con 0x555c03192b90 2024-08-01T04:25:37.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c031b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa876a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c031b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c03194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa876a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c031b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:45528/0 (socket says 172.21.15.3:45528) 2024-08-01T04:25:37.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c03194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43044/0 (socket says 172.21.15.3:43044) 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c031b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56060/0 (socket says 172.21.15.3:56060) 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa876a00640 1 -- 172.21.15.3:0/513010769 learned_addr learned my addr 172.21.15.3:0/513010769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 -- 172.21.15.3:0/513010769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 msgr2=0x555c031b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c031b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 -- 172.21.15.3:0/513010769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 msgr2=0x555c031b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c031b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa877400640 1 -- 172.21.15.3:0/513010769 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0318ef50 con 0x555c03192b90 2024-08-01T04:25:37.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.071+0000 7fa876a00640 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c031b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:25:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.072+0000 7fa877400640 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c03194d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8680ed0b0 tx=0x7fa8680f34e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d74daa28faa650d server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.072+0000 7fa863e00640 1 -- 172.21.15.3:0/513010769 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868049040 con 0x555c03192b90 2024-08-01T04:25:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.072+0000 7fa863e00640 1 -- 172.21.15.3:0/513010769 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8680bbff0 con 0x555c03192b90 2024-08-01T04:25:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.072+0000 7fa863e00640 1 -- 172.21.15.3:0/513010769 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680b8660 con 0x555c03192b90 2024-08-01T04:25:37.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 -- 172.21.15.3:0/513010769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 msgr2=0x555c03194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c03194d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8680ed0b0 tx=0x7fa8680f34e0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 -- 172.21.15.3:0/513010769 shutdown_connections 2024-08-01T04:25:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c0319dd70 0x555c031b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c031934b0 0x555c031b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 --2- 172.21.15.3:0/513010769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c03194d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 -- 172.21.15.3:0/513010769 >> 172.21.15.3:0/513010769 conn(0x555c0318f120 msgr2=0x555c03194630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 -- 172.21.15.3:0/513010769 shutdown_connections 2024-08-01T04:25:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.073+0000 7fa880fe0740 1 -- 172.21.15.3:0/513010769 wait complete. 2024-08-01T04:25:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 -- start start 2024-08-01T04:25:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 Processor -- start 2024-08-01T04:25:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c031a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 0x555c03197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0319dd70 0x555c03198200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ae0c0 con 0x555c031934b0 2024-08-01T04:25:37.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781af530 con 0x555c0319dd70 2024-08-01T04:25:37.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ae790 con 0x555c03192b90 2024-08-01T04:25:37.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.074+0000 7fa880fe0740 10 client.0 _ll_register_callbacks cb 0x7fa86401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:25:37.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 0x555c03197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa877400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c031a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa877400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c031a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43056/0 (socket says 172.21.15.3:43056) 2024-08-01T04:25:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa877400640 1 -- 172.21.15.3:0/2937580186 learned_addr learned my addr 172.21.15.3:0/2937580186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:25:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 0x555c03197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56064/0 (socket says 172.21.15.3:56064) 2024-08-01T04:25:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa877e00640 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0319dd70 0x555c03198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 -- 172.21.15.3:0/2937580186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 msgr2=0x555c031a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c031a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 -- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0319dd70 msgr2=0x555c03198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0319dd70 0x555c03198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.075+0000 7fa876a00640 1 -- 172.21.15.3:0/2937580186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c031902b0 con 0x555c031934b0 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa876a00640 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 0x555c03197ac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa864004b60 tx=0x7fa864005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a9fa005212ecc2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa861600640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:25:37.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa861600640 1 -- 172.21.15.3:0/2937580186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86401e040 con 0x555c031934b0 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa861600640 1 -- 172.21.15.3:0/2937580186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8629+0+0 (secure 0 0 0) 0x7fa8640035f0 con 0x555c031934b0 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8680bbff0 con 0x555c031934b0 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa861600640 1 -- 172.21.15.3:0/2937580186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa864046050 con 0x555c031934b0 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa880fe0740 10 client.16182 fetch_fsmap learned FSMap version 29 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.076+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c032184d0 con 0x555c031934b0 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa861600640 1 -- 172.21.15.3:0/2937580186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8680bbff0 con 0x555c031934b0 2024-08-01T04:25:37.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa861600640 1 -- 172.21.15.3:0/2937580186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa864004370 con 0x555c031934b0 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa880fe0740 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa880fe0740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa880fe0740 20 client.16182 populate_metadata read hostname 'smithi003' 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c03218c40 0x555c0321b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa880fe0740 4 client.16182 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c03100670 con 0x555c03218c40 2024-08-01T04:25:37.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.077+0000 7fa877400640 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c03218c40 0x555c0321b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.078+0000 7fa877400640 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c03218c40 0x555c0321b060 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad50806c4b14acb server_cookie=22e7ceb035c675fa in_seq=0 out_seq=0 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.078+0000 7fa861600640 10 client.16182 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.078+0000 7fa861600640 1 -- 172.21.15.3:0/2937580186 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555c03100670 con 0x555c03218c40 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.078+0000 7fa861600640 10 client.16182 handle_command_reply: tid=0 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 client.16182 shutdown 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 msgr2=0x555c03197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 0x555c03197ac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa864004b60 tx=0x7fa864005990 comp rx=0 tx=0).stop 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 shutdown_connections 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x555c03218c40 0x555c0321b060 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555c0319dd70 0x555c03198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555c031934b0 0x555c03197ac0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 --2- 172.21.15.3:0/2937580186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555c03192b90 0x555c031a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 >> 172.21.15.3:0/2937580186 conn(0x555c0318f120 msgr2=0x555c0318e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa861600640 0 client.16182 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:25:37.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 shutdown_connections 2024-08-01T04:25:37.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 1 -- 172.21.15.3:0/2937580186 wait complete. 2024-08-01T04:25:37.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.079+0000 7fa880fe0740 20 client.16182 trim_cache size 0 max 16384 2024-08-01T04:25:37.208 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:25:37.208 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-01T04:25:37.208 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout: "171d234e-d449-4740-bf30-ed5e0e095c78": { 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "171d234e-d449-4740-bf30-ed5e0e095c78", 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:25:37.209 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa85c0a2d70 msgr2=0x7fa85c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa85c0a2d70 0x7fa85c0a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa870009fd0 tx=0x7fa870009500 comp rx=0 tx=0).stop 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 msgr2=0x7fa87819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 0x7fa87819a3e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa868004770 tx=0x7fa868003c10 comp rx=0 tx=0).stop 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 shutdown_connections 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fa85c0a2d70 0x7fa85c0a5230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa87810e160 0x7fa87819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa878074c10 0x7fa87819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 --2- 172.21.15.3:0/3702665041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8780742f0 0x7fa87819a3e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:25:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 >> 172.21.15.3:0/3702665041 conn(0x7fa8780fd5e0 msgr2=0x7fa8780fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:25:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 shutdown_connections 2024-08-01T04:25:37.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:25:37.080+0000 7fa87e600640 1 -- 172.21.15.3:0/3702665041 wait complete. 2024-08-01T04:25:37.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:37 smithi160 ceph-mon[37224]: pgmap v289: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:38.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:37 smithi003 ceph-mon[32262]: pgmap v289: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:38.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:37 smithi163 ceph-mon[37383]: pgmap v289: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:25:38.610 INFO:tasks.cephfs.filesystem:scrub status for tag:171d234e-d449-4740-bf30-ed5e0e095c78 - {'path': '/', 'tag': '171d234e-d449-4740-bf30-ed5e0e095c78', 'options': 'recursive,force'} 2024-08-01T04:25:39.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:39 smithi160 ceph-mon[37224]: pgmap v290: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:40.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:39 smithi163 ceph-mon[37383]: pgmap v290: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:40.208 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:39 smithi003 ceph-mon[32262]: pgmap v290: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:41.903 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:41 smithi160 ceph-mon[37224]: pgmap v291: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:25:42.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:41 smithi163 ceph-mon[37383]: pgmap v291: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:25:42.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:41 smithi003 ceph-mon[32262]: pgmap v291: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:25:42.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:42 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:25:42.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:42 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:25:42.904 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:42 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:25:43.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:42 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:25:43.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:42 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:25:43.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:42 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:25:43.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:42 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:25:43.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:42 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:25:43.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:42 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:25:43.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:43 smithi160 ceph-mon[37224]: pgmap v292: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:25:44.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:43 smithi163 ceph-mon[37383]: pgmap v292: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:25:44.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:43 smithi003 ceph-mon[32262]: pgmap v292: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:25:45.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:45 smithi160 ceph-mon[37224]: pgmap v293: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:46.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:45 smithi163 ceph-mon[37383]: pgmap v293: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:46.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:45 smithi003 ceph-mon[32262]: pgmap v293: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:25:47.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:47 smithi003 ceph-mon[32262]: pgmap v294: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:25:47.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:47 smithi160 ceph-mon[37224]: pgmap v294: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:25:48.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:47 smithi163 ceph-mon[37383]: pgmap v294: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:25:49.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:49 smithi160 ceph-mon[37224]: pgmap v295: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:25:50.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:49 smithi163 ceph-mon[37383]: pgmap v295: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:25:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:49 smithi003 ceph-mon[32262]: pgmap v295: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:25:51.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:51 smithi160 ceph-mon[37224]: pgmap v296: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:25:52.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:51 smithi163 ceph-mon[37383]: pgmap v296: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:25:52.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:51 smithi003 ceph-mon[32262]: pgmap v296: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:25:53.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:53 smithi160 ceph-mon[37224]: pgmap v297: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:54.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:53 smithi163 ceph-mon[37383]: pgmap v297: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:54.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:53 smithi003 ceph-mon[32262]: pgmap v297: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:55.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:55 smithi160 ceph-mon[37224]: pgmap v298: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:25:56.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:55 smithi163 ceph-mon[37383]: pgmap v298: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:25:56.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:55 smithi003 ceph-mon[32262]: pgmap v298: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:25:58.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:57 smithi163 ceph-mon[37383]: pgmap v299: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:58.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:57 smithi003 ceph-mon[32262]: pgmap v299: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:25:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:57 smithi160 ceph-mon[37224]: pgmap v299: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:00.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:25:59 smithi163 ceph-mon[37383]: pgmap v300: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:26:00.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:25:59 smithi003 ceph-mon[32262]: pgmap v300: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:26:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:25:59 smithi160 ceph-mon[37224]: pgmap v300: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:26:02.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:01 smithi163 ceph-mon[37383]: pgmap v301: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:26:02.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:01 smithi003 ceph-mon[32262]: pgmap v301: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:26:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:01 smithi160 ceph-mon[37224]: pgmap v301: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:26:04.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:03 smithi163 ceph-mon[37383]: pgmap v302: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:26:04.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:03 smithi003 ceph-mon[32262]: pgmap v302: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:26:04.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:03 smithi160 ceph-mon[37224]: pgmap v302: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:26:06.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:05 smithi163 ceph-mon[37383]: pgmap v303: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:26:06.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:05 smithi003 ceph-mon[32262]: pgmap v303: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:26:06.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:05 smithi160 ceph-mon[37224]: pgmap v303: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:26:07.956 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:07 smithi160 ceph-mon[37224]: pgmap v304: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:26:07.957 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:07 smithi160 ceph-mon[37224]: osdmap e98: 12 total, 12 up, 12 in 2024-08-01T04:26:08.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:07 smithi163 ceph-mon[37383]: pgmap v304: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:26:08.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:07 smithi163 ceph-mon[37383]: osdmap e98: 12 total, 12 up, 12 in 2024-08-01T04:26:08.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:07 smithi003 ceph-mon[32262]: pgmap v304: 97 pgs: 97 active+clean; 24 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:26:08.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:07 smithi003 ceph-mon[32262]: osdmap e98: 12 total, 12 up, 12 in 2024-08-01T04:26:08.751 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:26:10.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:09 smithi163 ceph-mon[37383]: pgmap v306: 97 pgs: 3 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 92 active+clean; 23 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:26:10.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:09 smithi003 ceph-mon[32262]: pgmap v306: 97 pgs: 3 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 92 active+clean; 23 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:26:10.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:09 smithi160 ceph-mon[37224]: pgmap v306: 97 pgs: 3 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 92 active+clean; 23 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:26:10.214 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:12.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:11 smithi163 ceph-mon[37383]: pgmap v307: 97 pgs: 9 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 76 active+clean; 23 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:26:12.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:11 smithi003 ceph-mon[32262]: pgmap v307: 97 pgs: 9 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 76 active+clean; 23 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:26:12.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:11 smithi160 ceph-mon[37224]: pgmap v307: 97 pgs: 9 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 76 active+clean; 23 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:26:14.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:13 smithi163 ceph-mon[37383]: pgmap v308: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 21 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:26:14.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:13 smithi003 ceph-mon[32262]: pgmap v308: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 21 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:26:14.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:13 smithi160 ceph-mon[37224]: pgmap v308: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 21 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-08-01T04:26:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:15 smithi163 ceph-mon[37383]: pgmap v309: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 21 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:26:16.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:15 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:26:16.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:15 smithi003 ceph-mon[32262]: pgmap v309: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 21 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:26:16.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:15 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:26:16.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:15 smithi160 ceph-mon[37224]: pgmap v309: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 21 GiB data, 51 GiB used, 1021 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:26:16.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:15 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:26:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 -- 172.21.15.3:0/4137743292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884076450 msgr2=0x7f1884076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 --2- 172.21.15.3:0/4137743292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884076450 0x7f1884076830 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1874009900 tx=0x7f187402a410 comp rx=0 tx=0).stop 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 -- 172.21.15.3:0/4137743292 shutdown_connections 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 --2- 172.21.15.3:0/4137743292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884112760 0x7f1884114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 --2- 172.21.15.3:0/4137743292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076d70 0x7f1884112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 --2- 172.21.15.3:0/4137743292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884076450 0x7f1884076830 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 -- 172.21.15.3:0/4137743292 >> 172.21.15.3:0/4137743292 conn(0x7f18840ff7c0 msgr2=0x7f1884101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 -- 172.21.15.3:0/4137743292 shutdown_connections 2024-08-01T04:26:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.251+0000 7f1888a00640 1 -- 172.21.15.3:0/4137743292 wait complete. 2024-08-01T04:26:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 Processor -- start 2024-08-01T04:26:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 -- start start 2024-08-01T04:26:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076450 0x7f18841a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 0x7f18841a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884112760 0x7f188419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18841182d0 con 0x7f1884112760 2024-08-01T04:26:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1884118150 con 0x7f1884076d70 2024-08-01T04:26:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.253+0000 7f1888a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f1884118450 con 0x7f1884076450 2024-08-01T04:26:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 0x7f18841a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076450 0x7f18841a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 0x7f18841a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39350/0 (socket says 172.21.15.3:39350) 2024-08-01T04:26:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076450 0x7f18841a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51730/0 (socket says 172.21.15.3:51730) 2024-08-01T04:26:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 -- 172.21.15.3:0/2100294707 learned_addr learned my addr 172.21.15.3:0/2100294707 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1882800640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884112760 0x7f188419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 -- 172.21.15.3:0/2100294707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076450 msgr2=0x7f18841a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076450 0x7f18841a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 -- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884112760 msgr2=0x7f188419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884112760 0x7f188419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.254+0000 7f1881400640 1 -- 172.21.15.3:0/2100294707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188419ecf0 con 0x7f1884076d70 2024-08-01T04:26:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.255+0000 7f1881400640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 0x7f18841a4940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f187000e9f0 tx=0x7f187000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.255+0000 7f187aa00640 1 -- 172.21.15.3:0/2100294707 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000ccc0 con 0x7f1884076d70 2024-08-01T04:26:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.255+0000 7f187aa00640 1 -- 172.21.15.3:0/2100294707 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1870010910 con 0x7f1884076d70 2024-08-01T04:26:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.255+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188419efe0 con 0x7f1884076d70 2024-08-01T04:26:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.255+0000 7f187aa00640 1 -- 172.21.15.3:0/2100294707 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700188a0 con 0x7f1884076d70 2024-08-01T04:26:17.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.255+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841ab2d0 con 0x7f1884076d70 2024-08-01T04:26:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.258+0000 7f187aa00640 1 -- 172.21.15.3:0/2100294707 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1870010ac0 con 0x7f1884076d70 2024-08-01T04:26:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.260+0000 7f187aa00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18680a2d70 0x7f18680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.260+0000 7f187aa00640 1 -- 172.21.15.3:0/2100294707 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8673+0+0 (secure 0 0 0) 0x7f18700cf900 con 0x7f1884076d70 2024-08-01T04:26:17.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.260+0000 7f1881e00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18680a2d70 0x7f18680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.261+0000 7f1881e00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18680a2d70 0x7f18680a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1874009230 tx=0x7f18740058d0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.283+0000 7f188b680740 1 Processor -- start 2024-08-01T04:26:17.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.283+0000 7f188b680740 1 -- start start 2024-08-01T04:26:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.283+0000 7f188b680740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df40f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.283+0000 7f188b680740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df4b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f188b680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df4e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f188b680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e43dea6bf0 con 0x55e43df4bd70 2024-08-01T04:26:17.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f188b680740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e43de3ebd0 con 0x55e43df414b0 2024-08-01T04:26:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f188b680740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e43dea2bb0 con 0x55e43df40b90 2024-08-01T04:26:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f1881e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df40f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f1881e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df40f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51740/0 (socket says 172.21.15.3:51740) 2024-08-01T04:26:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f1881400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df4b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f1882800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df4e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.284+0000 7f1881e00640 1 -- 172.21.15.3:0/474983397 learned_addr learned my addr 172.21.15.3:0/474983397 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:17.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f1881400640 1 -- 172.21.15.3:0/474983397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 msgr2=0x55e43df40f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f1881400640 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df40f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f1881400640 1 -- 172.21.15.3:0/474983397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 msgr2=0x55e43df4e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f1881400640 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df4e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f1881400640 1 -- 172.21.15.3:0/474983397 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43de458a0 con 0x55e43df414b0 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f1881400640 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df4b830 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f18700ccf50 tx=0x7f187009b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a1e186a133bfb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f186f000640 1 -- 172.21.15.3:0/474983397 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700a4990 con 0x55e43df414b0 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f186f000640 1 -- 172.21.15.3:0/474983397 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18700cd1d0 con 0x55e43df414b0 2024-08-01T04:26:17.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.285+0000 7f186f000640 1 -- 172.21.15.3:0/474983397 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870097cd0 con 0x55e43df414b0 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 -- 172.21.15.3:0/474983397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 msgr2=0x55e43df4b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df4b830 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f18700ccf50 tx=0x7f187009b970 comp rx=0 tx=0).stop 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 -- 172.21.15.3:0/474983397 shutdown_connections 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df4e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df4b830 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 --2- 172.21.15.3:0/474983397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df40f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.286+0000 7f188b680740 1 -- 172.21.15.3:0/474983397 >> 172.21.15.3:0/474983397 conn(0x55e43df3d120 msgr2=0x55e43df3d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:17.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.287+0000 7f188b680740 1 -- 172.21.15.3:0/474983397 shutdown_connections 2024-08-01T04:26:17.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.287+0000 7f188b680740 1 -- 172.21.15.3:0/474983397 wait complete. 2024-08-01T04:26:17.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.287+0000 7f188b680740 1 -- start start 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.287+0000 7f188b680740 1 Processor -- start 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df48c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df49700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e43df4e940 con 0x55e43df4bd70 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e43de61d40 con 0x55e43df414b0 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e43df4eac0 con 0x55e43df40b90 2024-08-01T04:26:17.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f188b680740 10 client.0 _ll_register_callbacks cb 0x55e43de2ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f1882800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df49700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f1881e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df48c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f1881400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f1882800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df49700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45828/0 (socket says 172.21.15.3:45828) 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f1882800640 1 -- 172.21.15.3:0/464427134 learned_addr learned my addr 172.21.15.3:0/464427134 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.288+0000 7f1881e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df48c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51748/0 (socket says 172.21.15.3:51748) 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1881400640 1 -- 172.21.15.3:0/464427134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 msgr2=0x55e43df48c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1881400640 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df48c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1881400640 1 -- 172.21.15.3:0/464427134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 msgr2=0x55e43df49700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1881400640 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df49700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1881400640 1 -- 172.21.15.3:0/464427134 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43dc183c0 con 0x55e43df414b0 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1881400640 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df491c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e43df41ad0 tx=0x7f1870009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209574abe5e820cb server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.289+0000 7f1863400640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f1863400640 1 -- 172.21.15.3:0/464427134 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700b9070 con 0x55e43df414b0 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f1863400640 1 -- 172.21.15.3:0/464427134 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8673+0+0 (secure 0 0 0) 0x7f1870010640 con 0x55e43df414b0 2024-08-01T04:26:17.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18700cd1d0 con 0x55e43df414b0 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f1863400640 1 -- 172.21.15.3:0/464427134 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f187009b170 con 0x55e43df414b0 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f188b680740 10 client.26381 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e43df5a620 con 0x55e43df414b0 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.290+0000 7f1863400640 1 -- 172.21.15.3:0/464427134 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18700cd1d0 con 0x55e43df414b0 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f1863400640 1 -- 172.21.15.3:0/464427134 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f187009ad70 con 0x55e43df414b0 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f188b680740 10 client.26381 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f188b680740 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:17.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f188b680740 20 client.26381 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f188b680740 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43df5b970 0x55e43df5dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f188b680740 4 client.26381 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:17.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e43deb4160 con 0x55e43df5b970 2024-08-01T04:26:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.291+0000 7f1881e00640 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43df5b970 0x55e43df5dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.292+0000 7f1881e00640 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43df5b970 0x55e43df5dd60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7def634beaf578 server_cookie=94f131737a08ca35 in_seq=0 out_seq=0 2024-08-01T04:26:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.292+0000 7f1863400640 10 client.26381 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f1863400640 1 -- 172.21.15.3:0/464427134 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e43deb4160 con 0x55e43df5b970 2024-08-01T04:26:17.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f1863400640 10 client.26381 handle_command_reply: tid=0 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 client.26381 shutdown 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 msgr2=0x55e43df491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df491c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55e43df41ad0 tx=0x7f1870009700 comp rx=0 tx=0).stop 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 shutdown_connections 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43df5b970 0x55e43df5dd60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df49700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df414b0 0x55e43df491c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 --2- 172.21.15.3:0/464427134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df40b90 0x55e43df48c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 >> 172.21.15.3:0/464427134 conn(0x55e43df3d120 msgr2=0x55e43df42e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.293+0000 7f1863400640 0 client.26381 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.294+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 shutdown_connections 2024-08-01T04:26:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.294+0000 7f188b680740 1 -- 172.21.15.3:0/464427134 wait complete. 2024-08-01T04:26:17.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.294+0000 7f188b680740 20 client.26381 trim_cache size 0 max 16384 2024-08-01T04:26:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.313+0000 7f188b680740 1 Processor -- start 2024-08-01T04:26:17.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.313+0000 7f188b680740 1 -- start start 2024-08-01T04:26:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f188b680740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 0x55e43df49550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f188b680740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df45ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f188b680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df46420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f188b680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18841b6710 con 0x55e43df4bd70 2024-08-01T04:26:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f188b680740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f18841b3fe0 con 0x55e43df40b90 2024-08-01T04:26:17.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f188b680740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f18841b1e50 con 0x55e43df414b0 2024-08-01T04:26:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f1881e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 0x55e43df49550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f1882800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df46420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f1881e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 0x55e43df49550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:39370/0 (socket says 172.21.15.3:39370) 2024-08-01T04:26:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f1881e00640 1 -- 172.21.15.3:0/1478658335 learned_addr learned my addr 172.21.15.3:0/1478658335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.314+0000 7f1881400640 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df45ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881e00640 1 -- 172.21.15.3:0/1478658335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 msgr2=0x55e43df45ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881e00640 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df45ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881e00640 1 -- 172.21.15.3:0/1478658335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 msgr2=0x55e43df46420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881e00640 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df46420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881e00640 1 -- 172.21.15.3:0/1478658335 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43df412e0 con 0x55e43df40b90 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881400640 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df45ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f1881e00640 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 0x55e43df49550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1874033000 tx=0x7f1874040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a9aeff1599bfb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f186e600640 1 -- 172.21.15.3:0/1478658335 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874059070 con 0x55e43df40b90 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f186e600640 1 -- 172.21.15.3:0/1478658335 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874003ad0 con 0x55e43df40b90 2024-08-01T04:26:17.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.315+0000 7f186e600640 1 -- 172.21.15.3:0/1478658335 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874035040 con 0x55e43df40b90 2024-08-01T04:26:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 -- 172.21.15.3:0/1478658335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 msgr2=0x55e43df49550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 0x55e43df49550 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1874033000 tx=0x7f1874040000 comp rx=0 tx=0).stop 2024-08-01T04:26:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 -- 172.21.15.3:0/1478658335 shutdown_connections 2024-08-01T04:26:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df4bd70 0x55e43df46420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df45ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 --2- 172.21.15.3:0/1478658335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df40b90 0x55e43df49550 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.316+0000 7f188b680740 1 -- 172.21.15.3:0/1478658335 >> 172.21.15.3:0/1478658335 conn(0x55e43df3d120 msgr2=0x55e43df4ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.317+0000 7f188b680740 1 -- 172.21.15.3:0/1478658335 shutdown_connections 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.317+0000 7f188b680740 1 -- 172.21.15.3:0/1478658335 wait complete. 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.317+0000 7f188b680740 1 -- start start 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.317+0000 7f188b680740 1 Processor -- start 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df40b90 0x55e43df5dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df5e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df4bd70 0x55e43dfc42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18841b24a0 con 0x55e43df40b90 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f18841b3910 con 0x55e43df4bd70 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f18841b2b70 con 0x55e43df414b0 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f188b680740 10 client.0 _ll_register_callbacks cb 0x7f187c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f1881400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df5e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f1881400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df5e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:51760/0 (socket says 172.21.15.3:51760) 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f1881400640 1 -- 172.21.15.3:0/119339680 learned_addr learned my addr 172.21.15.3:0/119339680 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:17.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f1881e00640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df40b90 0x55e43df5dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.318+0000 7f1882800640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df4bd70 0x55e43dfc42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1881400640 1 -- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df4bd70 msgr2=0x55e43dfc42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1881400640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df4bd70 0x55e43dfc42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1881400640 1 -- 172.21.15.3:0/119339680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df40b90 msgr2=0x55e43df5dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1881400640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df40b90 0x55e43df5dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1881400640 1 -- 172.21.15.3:0/119339680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43dfc4890 con 0x55e43df414b0 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1881400640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df5e400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1870098180 tx=0x7f18700cd510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4b7bf804442886 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:17.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1863400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f1863400640 1 -- 172.21.15.3:0/119339680 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18700b9040 con 0x55e43df414b0 2024-08-01T04:26:17.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.319+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1874003ad0 con 0x55e43df414b0 2024-08-01T04:26:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f1863400640 1 -- 172.21.15.3:0/119339680 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8673+0+0 (secure 0 0 0) 0x7f187009abe0 con 0x55e43df414b0 2024-08-01T04:26:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f1863400640 1 -- 172.21.15.3:0/119339680 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f187009b680 con 0x55e43df414b0 2024-08-01T04:26:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f188b680740 10 client.26359 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e43dfc5b60 con 0x55e43df414b0 2024-08-01T04:26:17.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f1863400640 1 -- 172.21.15.3:0/119339680 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874003ad0 con 0x55e43df414b0 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f1863400640 1 -- 172.21.15.3:0/119339680 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f18700a4be0 con 0x55e43df414b0 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f188b680740 10 client.26359 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.320+0000 7f188b680740 10 client.26359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.321+0000 7f188b680740 20 client.26359 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.321+0000 7f188b680740 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43dfc5fe0 0x55e43dfc8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.321+0000 7f188b680740 4 client.26359 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.321+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e43deae670 con 0x55e43dfc5fe0 2024-08-01T04:26:17.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.321+0000 7f1881e00640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43dfc5fe0 0x55e43dfc8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f1881e00640 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43dfc5fe0 0x55e43dfc8400 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267e03bb9c56b17b server_cookie=eff82b8b5c08c11c in_seq=0 out_seq=0 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f1863400640 10 client.26359 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f1863400640 1 -- 172.21.15.3:0/119339680 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e43deae670 con 0x55e43dfc5fe0 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f1863400640 10 client.26359 handle_command_reply: tid=0 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 client.26359 shutdown 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 msgr2=0x55e43df5e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df5e400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1870098180 tx=0x7f18700cd510 comp rx=0 tx=0).stop 2024-08-01T04:26:17.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 shutdown_connections 2024-08-01T04:26:17.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e43dfc5fe0 0x55e43dfc8400 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e43df4bd70 0x55e43dfc42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e43df414b0 0x55e43df5e400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 --2- 172.21.15.3:0/119339680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e43df40b90 0x55e43df5dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f1863400640 0 client.26359 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:17.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.322+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 >> 172.21.15.3:0/119339680 conn(0x55e43df3d120 msgr2=0x55e43df47420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:17.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.323+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 shutdown_connections 2024-08-01T04:26:17.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.323+0000 7f188b680740 1 -- 172.21.15.3:0/119339680 wait complete. 2024-08-01T04:26:17.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.323+0000 7f188b680740 20 client.26359 trim_cache size 0 max 16384 2024-08-01T04:26:17.331 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:17.331 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:26:17.332 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:26:17.332 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:17.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18680a2d70 msgr2=0x7f18680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18680a2d70 0x7f18680a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1874009230 tx=0x7f18740058d0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 msgr2=0x7f18841a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:17.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 0x7f18841a4940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f187000e9f0 tx=0x7f187000edf0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 shutdown_connections 2024-08-01T04:26:17.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1884112760 0x7f188419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1884076d70 0x7f18841a4940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f18680a2d70 0x7f18680a5230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 --2- 172.21.15.3:0/2100294707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884076450 0x7f18841a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:17.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 >> 172.21.15.3:0/2100294707 conn(0x7f18840ff7c0 msgr2=0x7f1884101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:17.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 shutdown_connections 2024-08-01T04:26:17.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:17.324+0000 7f1888a00640 1 -- 172.21.15.3:0/2100294707 wait complete. 2024-08-01T04:26:17.908 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:17 smithi003 ceph-mon[32262]: pgmap v310: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 19 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:26:17.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:17 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:26:17.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:17 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:26:17.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:17 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:26:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:17 smithi163 ceph-mon[37383]: pgmap v310: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 19 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:26:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:17 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:26:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:17 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:26:18.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:17 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:26:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:17 smithi160 ceph-mon[37224]: pgmap v310: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 19 GiB data, 50 GiB used, 1023 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:26:18.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:17 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:26:18.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:17 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:26:18.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:17 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:26:18.535 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:26:18.536 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:26:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:19.743 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:19 smithi003 ceph-mon[32262]: pgmap v311: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 178 B/s wr, 6 op/s 2024-08-01T04:26:19.744 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:19 smithi003 ceph-mon[32262]: osdmap e99: 12 total, 12 up, 12 in 2024-08-01T04:26:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.773+0000 7f72a6400640 1 -- 172.21.15.3:0/1590638401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 msgr2=0x7f72a0109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.773+0000 7f72a6400640 1 --2- 172.21.15.3:0/1590638401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 0x7f72a0109ce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7294009900 tx=0x7f729402a410 comp rx=0 tx=0).stop 2024-08-01T04:26:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 -- 172.21.15.3:0/1590638401 shutdown_connections 2024-08-01T04:26:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 --2- 172.21.15.3:0/1590638401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a01107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 --2- 172.21.15.3:0/1590638401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a0103900 0x7f72a0103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 --2- 172.21.15.3:0/1590638401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 0x7f72a0109ce0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 -- 172.21.15.3:0/1590638401 >> 172.21.15.3:0/1590638401 conn(0x7f72a00ff640 msgr2=0x7f72a0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 -- 172.21.15.3:0/1590638401 shutdown_connections 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.774+0000 7f72a6400640 1 -- 172.21.15.3:0/1590638401 wait complete. 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.775+0000 7f72a6400640 1 Processor -- start 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.775+0000 7f72a6400640 1 -- start start 2024-08-01T04:26:19.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a6400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a0103900 0x7f72a019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a6400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a6400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 0x7f72a019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a6400640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0113d20 con 0x7f72a01042a0 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a6400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0113ba0 con 0x7f72a0109900 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a6400640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0113ea0 con 0x7f72a0103900 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f729fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f729fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45866/0 (socket says 172.21.15.3:45866) 2024-08-01T04:26:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f729fe00640 1 -- 172.21.15.3:0/2516290169 learned_addr learned my addr 172.21.15.3:0/2516290169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a5000640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a0103900 0x7f72a019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.776+0000 7f72a5a00640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 0x7f72a019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729fe00640 1 -- 172.21.15.3:0/2516290169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a0103900 msgr2=0x7f72a019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729fe00640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a0103900 0x7f72a019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729fe00640 1 -- 172.21.15.3:0/2516290169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 msgr2=0x7f72a019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729fe00640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 0x7f72a019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729fe00640 1 -- 172.21.15.3:0/2516290169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019f1b0 con 0x7f72a01042a0 2024-08-01T04:26:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729fe00640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a019a740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f728c00ea60 tx=0x7f728c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.777+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c00cde0 con 0x7f72a01042a0 2024-08-01T04:26:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.778+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019f4a0 con 0x7f72a01042a0 2024-08-01T04:26:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.778+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728c0035f0 con 0x7f72a01042a0 2024-08-01T04:26:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.778+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c010670 con 0x7f72a01042a0 2024-08-01T04:26:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.778+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a01a6d80 con 0x7f72a01042a0 2024-08-01T04:26:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.779+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a0105040 con 0x7f72a01042a0 2024-08-01T04:26:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.781+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f728c0037a0 con 0x7f72a01042a0 2024-08-01T04:26:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.784+0000 7f729d600640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f72880a2d70 0x7f72880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.784+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f728c0cf7b0 con 0x7f72a01042a0 2024-08-01T04:26:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.785+0000 7f72a5000640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f72880a2d70 0x7f72880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.786+0000 7f72a5000640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f72880a2d70 0x7f72880a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7294000c00 tx=0x7f7294005b50 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:19.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:19.788+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f728c09c300 con 0x7f72a01042a0 2024-08-01T04:26:20.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.031+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72a0103d60 con 0x7f72a01042a0 2024-08-01T04:26:20.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.033+0000 7f729d600640 1 -- 172.21.15.3:0/2516290169 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f728c09bc00 con 0x7f72a01042a0 2024-08-01T04:26:20.035 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:26:20.036 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:26:20.036 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:26:20.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f72880a2d70 msgr2=0x7f72880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:20.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f72880a2d70 0x7f72880a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7294000c00 tx=0x7f7294005b50 comp rx=0 tx=0).stop 2024-08-01T04:26:20.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 msgr2=0x7f72a019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:20.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a019a740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f728c00ea60 tx=0x7f728c00ef20 comp rx=0 tx=0).stop 2024-08-01T04:26:20.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 shutdown_connections 2024-08-01T04:26:20.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72a0109900 0x7f72a019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:20.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72a01042a0 0x7f72a019a740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:20.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f72880a2d70 0x7f72880a5230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:20.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 --2- 172.21.15.3:0/2516290169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a0103900 0x7f72a019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:20.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 >> 172.21.15.3:0/2516290169 conn(0x7f72a00ff640 msgr2=0x7f72a00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:20.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.037+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 shutdown_connections 2024-08-01T04:26:20.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:20.038+0000 7f72a6400640 1 -- 172.21.15.3:0/2516290169 wait complete. 2024-08-01T04:26:20.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:19 smithi163 ceph-mon[37383]: pgmap v311: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 178 B/s wr, 6 op/s 2024-08-01T04:26:20.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:19 smithi163 ceph-mon[37383]: osdmap e99: 12 total, 12 up, 12 in 2024-08-01T04:26:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:19 smithi160 ceph-mon[37224]: pgmap v311: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.7 KiB/s rd, 178 B/s wr, 6 op/s 2024-08-01T04:26:20.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:19 smithi160 ceph-mon[37224]: osdmap e99: 12 total, 12 up, 12 in 2024-08-01T04:26:21.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:20 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2516290169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:26:21.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:20 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2516290169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:26:21.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:26:21.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:20 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2516290169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:26:21.589 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:21.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:21 smithi003 ceph-mon[32262]: pgmap v313: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:22.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:21 smithi163 ceph-mon[37383]: pgmap v313: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:22.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:21 smithi160 ceph-mon[37224]: pgmap v313: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 -- 172.21.15.3:0/3418864311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01016f0 msgr2=0x7f09a0101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 --2- 172.21.15.3:0/3418864311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01016f0 0x7f09a0101b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f098c009930 tx=0x7f098c02a420 comp rx=0 tx=0).stop 2024-08-01T04:26:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 -- 172.21.15.3:0/3418864311 shutdown_connections 2024-08-01T04:26:22.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 --2- 172.21.15.3:0/3418864311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09a0102090 0x7f09a010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 --2- 172.21.15.3:0/3418864311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a01016f0 0x7f09a0101b50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 --2- 172.21.15.3:0/3418864311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01076f0 0x7f09a0107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 -- 172.21.15.3:0/3418864311 >> 172.21.15.3:0/3418864311 conn(0x7f09a00fd470 msgr2=0x7f09a00ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 -- 172.21.15.3:0/3418864311 shutdown_connections 2024-08-01T04:26:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.273+0000 7f09a6200640 1 -- 172.21.15.3:0/3418864311 wait complete. 2024-08-01T04:26:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.275+0000 7f09a6200640 1 Processor -- start 2024-08-01T04:26:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.275+0000 7f09a6200640 1 -- start start 2024-08-01T04:26:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a6200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 0x7f09a019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a6200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0102090 0x7f09a019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a6200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09a01076f0 0x7f09a019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a6200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0111b40 con 0x7f09a0102090 2024-08-01T04:26:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a6200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01119c0 con 0x7f09a01016f0 2024-08-01T04:26:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a6200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0111cc0 con 0x7f09a01076f0 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a4e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 0x7f09a019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a4e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 0x7f09a019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50594/0 (socket says 172.21.15.3:50594) 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a4e00640 1 -- 172.21.15.3:0/2854346552 learned_addr learned my addr 172.21.15.3:0/2854346552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f099fe00640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0102090 0x7f09a019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.276+0000 7f09a5800640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09a01076f0 0x7f09a019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f09a4e00640 1 -- 172.21.15.3:0/2854346552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09a01076f0 msgr2=0x7f09a019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f09a4e00640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09a01076f0 0x7f09a019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f09a4e00640 1 -- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0102090 msgr2=0x7f09a019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f09a4e00640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0102090 0x7f09a019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f09a4e00640 1 -- 172.21.15.3:0/2854346552 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a019f180 con 0x7f09a01016f0 2024-08-01T04:26:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f09a4e00640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 0x7f09a019a1d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f099000e9f0 tx=0x7f099000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.277+0000 7f099d600640 1 -- 172.21.15.3:0/2854346552 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000cd80 con 0x7f09a01016f0 2024-08-01T04:26:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.278+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a019f470 con 0x7f09a01016f0 2024-08-01T04:26:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.278+0000 7f099d600640 1 -- 172.21.15.3:0/2854346552 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0990010910 con 0x7f09a01016f0 2024-08-01T04:26:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.278+0000 7f099d600640 1 -- 172.21.15.3:0/2854346552 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990018990 con 0x7f09a01016f0 2024-08-01T04:26:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.278+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a01a6d50 con 0x7f09a01016f0 2024-08-01T04:26:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.281+0000 7f09aa473740 1 Processor -- start 2024-08-01T04:26:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.281+0000 7f09aa473740 1 -- start start 2024-08-01T04:26:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.281+0000 7f099d600640 1 -- 172.21.15.3:0/2854346552 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0990010ac0 con 0x7f09a01016f0 2024-08-01T04:26:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.281+0000 7f09aa473740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ff3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.281+0000 7f09aa473740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff4000 0x55b276ffe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09aa473740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277000cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09aa473740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b276f58bf0 con 0x55b276ff36e0 2024-08-01T04:26:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09aa473740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b276ef0b20 con 0x55b276ff4000 2024-08-01T04:26:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09aa473740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b276f54bb0 con 0x55b276ffe8c0 2024-08-01T04:26:22.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ff3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ff3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50780/0 (socket says 172.21.15.3:50780) 2024-08-01T04:26:22.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 -- 172.21.15.3:0/1766942866 learned_addr learned my addr 172.21.15.3:0/1766942866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:22.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a4e00640 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff4000 0x55b276ffe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f099fe00640 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277000cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 -- 172.21.15.3:0/1766942866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 msgr2=0x55b277000cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277000cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 -- 172.21.15.3:0/1766942866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff4000 msgr2=0x55b276ffe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff4000 0x55b276ffe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.282+0000 7f09a5800640 1 -- 172.21.15.3:0/1766942866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b276ef77f0 con 0x55b276ff36e0 2024-08-01T04:26:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.283+0000 7f09a5800640 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ff3ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0998009fd0 tx=0x7f09980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cd23d324a50483 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.283+0000 7f0996000640 1 -- 172.21.15.3:0/1766942866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998004d20 con 0x55b276ff36e0 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.283+0000 7f0996000640 1 -- 172.21.15.3:0/1766942866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998004050 con 0x55b276ff36e0 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.283+0000 7f0996000640 1 -- 172.21.15.3:0/1766942866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998005280 con 0x55b276ff36e0 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 -- 172.21.15.3:0/1766942866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 msgr2=0x55b276ff3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ff3ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0998009fd0 tx=0x7f09980118d0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f099d600640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f09840a2cf0 0x7f09840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 -- 172.21.15.3:0/1766942866 shutdown_connections 2024-08-01T04:26:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277000cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff4000 0x55b276ffe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 --2- 172.21.15.3:0/1766942866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ff3ac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 -- 172.21.15.3:0/1766942866 >> 172.21.15.3:0/1766942866 conn(0x55b276fefc70 msgr2=0x55b276ff0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f099d600640 1 -- 172.21.15.3:0/2854346552 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f09900cf7b0 con 0x7f09a01016f0 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09a4e00640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f09840a2cf0 0x7f09840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.284+0000 7f09aa473740 1 -- 172.21.15.3:0/1766942866 shutdown_connections 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.285+0000 7f09aa473740 1 -- 172.21.15.3:0/1766942866 wait complete. 2024-08-01T04:26:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.285+0000 7f09aa473740 1 -- start start 2024-08-01T04:26:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.285+0000 7f09a4e00640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f09840a2cf0 0x7f09840a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55b276ff0ad0 tx=0x7f09900d5530 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:22.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.285+0000 7f09aa473740 1 Processor -- start 2024-08-01T04:26:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ffb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ffbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ffe8c0 0x55b276ffc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b277001490 con 0x55b276ff36e0 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b276f13cf0 con 0x55b276ffe8c0 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b277001610 con 0x55b276ff4000 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09aa473740 10 client.0 _ll_register_callbacks cb 0x55b276effe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ffb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f099fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ffbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f099fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ffbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43370/0 (socket says 172.21.15.3:43370) 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09a5800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ffb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50786/0 (socket says 172.21.15.3:50786) 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f099fe00640 1 -- 172.21.15.3:0/2213010867 learned_addr learned my addr 172.21.15.3:0/2213010867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.286+0000 7f09a5800640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ffe8c0 0x55b276ffc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f099fe00640 1 -- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ffe8c0 msgr2=0x55b276ffc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f099fe00640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ffe8c0 0x55b276ffc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f099fe00640 1 -- 172.21.15.3:0/2213010867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 msgr2=0x55b276ffb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f099fe00640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ffb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f099fe00640 1 -- 172.21.15.3:0/2213010867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b276cca3c0 con 0x55b276ff4000 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f09a5800640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ffe8c0 0x55b276ffc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f099fe00640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ffbcf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f098c033000 tx=0x7f098c0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9a96174b82225 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f098aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.287+0000 7f098aa00640 1 -- 172.21.15.3:0/2213010867 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c035040 con 0x55b276ff4000 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.288+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0998004050 con 0x55b276ff4000 2024-08-01T04:26:22.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.288+0000 7f098aa00640 1 -- 172.21.15.3:0/2213010867 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f098c005450 con 0x55b276ff4000 2024-08-01T04:26:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.288+0000 7f098aa00640 1 -- 172.21.15.3:0/2213010867 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f098c005650 con 0x55b276ff4000 2024-08-01T04:26:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.288+0000 7f09aa473740 10 client.26377 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.288+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b27700d190 con 0x55b276ff4000 2024-08-01T04:26:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.288+0000 7f098aa00640 1 -- 172.21.15.3:0/2213010867 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998004050 con 0x55b276ff4000 2024-08-01T04:26:22.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f098aa00640 1 -- 172.21.15.3:0/2213010867 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f098c0319f0 con 0x55b276ff4000 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09aa473740 10 client.26377 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09aa473740 10 client.26377 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09aa473740 20 client.26377 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09aa473740 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b27700e5a0 0x55b277010990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09aa473740 4 client.26377 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b276f66160 con 0x55b27700e5a0 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.289+0000 7f09a5800640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b27700e5a0 0x55b277010990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.290+0000 7f09a5800640 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b27700e5a0 0x55b277010990 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4555da26f97451fe server_cookie=3a4e5aadf30dff67 in_seq=0 out_seq=0 2024-08-01T04:26:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.290+0000 7f098aa00640 10 client.26377 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.290+0000 7f098aa00640 1 -- 172.21.15.3:0/2213010867 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b276f66160 con 0x55b27700e5a0 2024-08-01T04:26:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.290+0000 7f098aa00640 10 client.26377 handle_command_reply: tid=0 2024-08-01T04:26:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 client.26377 shutdown 2024-08-01T04:26:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 msgr2=0x55b276ffbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ffbcf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f098c033000 tx=0x7f098c0043b0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 shutdown_connections 2024-08-01T04:26:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ffe8c0 0x55b276ffc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b27700e5a0 0x55b277010990 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ffbcf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 --2- 172.21.15.3:0/2213010867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff36e0 0x55b276ffb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f098aa00640 0 client.26377 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 >> 172.21.15.3:0/2213010867 conn(0x55b276fefc70 msgr2=0x55b276ff59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 shutdown_connections 2024-08-01T04:26:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 1 -- 172.21.15.3:0/2213010867 wait complete. 2024-08-01T04:26:22.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.291+0000 7f09aa473740 20 client.26377 trim_cache size 0 max 16384 2024-08-01T04:26:22.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 Processor -- start 2024-08-01T04:26:22.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 -- start start 2024-08-01T04:26:22.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff4000 0x55b276feebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277016260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01b2120 con 0x55b276ff4000 2024-08-01T04:26:22.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01af9f0 con 0x55b276ff36e0 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.306+0000 7f09aa473740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01ad860 con 0x55b276ffe8c0 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f099fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff4000 0x55b276feebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f09a5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f09a5800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50612/0 (socket says 172.21.15.3:50612) 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f09a5800640 1 -- 172.21.15.3:0/4109916192 learned_addr learned my addr 172.21.15.3:0/4109916192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f09a5800640 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277016260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f099fe00640 1 -- 172.21.15.3:0/4109916192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 msgr2=0x55b277016260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f099fe00640 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277016260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f099fe00640 1 -- 172.21.15.3:0/4109916192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 msgr2=0x55b276ff6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f099fe00640 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f099fe00640 1 -- 172.21.15.3:0/4109916192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b276fef520 con 0x55b276ff4000 2024-08-01T04:26:22.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.307+0000 7f09a5800640 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.308+0000 7f099fe00640 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff4000 0x55b276feebb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f098c031e40 tx=0x7f098c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c0778194cb8bd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.308+0000 7f0995600640 1 -- 172.21.15.3:0/4109916192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c035030 con 0x55b276ff4000 2024-08-01T04:26:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.308+0000 7f0995600640 1 -- 172.21.15.3:0/4109916192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098c041070 con 0x55b276ff4000 2024-08-01T04:26:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.308+0000 7f0995600640 1 -- 172.21.15.3:0/4109916192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c031930 con 0x55b276ff4000 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 -- 172.21.15.3:0/4109916192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff4000 msgr2=0x55b276feebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff4000 0x55b276feebb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f098c031e40 tx=0x7f098c02a420 comp rx=0 tx=0).stop 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 -- 172.21.15.3:0/4109916192 shutdown_connections 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ffe8c0 0x55b277016260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ff4000 0x55b276feebb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 --2- 172.21.15.3:0/4109916192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 -- 172.21.15.3:0/4109916192 >> 172.21.15.3:0/4109916192 conn(0x55b276fefc70 msgr2=0x55b276fee4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 -- 172.21.15.3:0/4109916192 shutdown_connections 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 -- 172.21.15.3:0/4109916192 wait complete. 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 -- start start 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.309+0000 7f09aa473740 1 Processor -- start 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ff8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ffe8c0 0x55b276ff9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01adeb0 con 0x55b276ffe8c0 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01af320 con 0x55b276ff36e0 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01ae580 con 0x55b276ff4000 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09aa473740 10 client.0 _ll_register_callbacks cb 0x7f098c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f099fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ff8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f099fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ff8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43386/0 (socket says 172.21.15.3:43386) 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f099fe00640 1 -- 172.21.15.3:0/250534488 learned_addr learned my addr 172.21.15.3:0/250534488 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:22.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09a5800640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.310+0000 7f09a5800640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ffe8c0 0x55b276ff9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f099fe00640 1 -- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 msgr2=0x55b276ff86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f099fe00640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f099fe00640 1 -- 172.21.15.3:0/250534488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ffe8c0 msgr2=0x55b276ff9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f099fe00640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ffe8c0 0x55b276ff9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f099fe00640 1 -- 172.21.15.3:0/250534488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b277078230 con 0x55b276ff4000 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f099fe00640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ff8c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f098c031f30 tx=0x7f098c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c94a651b42ec21 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f098aa00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f098aa00640 1 -- 172.21.15.3:0/250534488 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c035040 con 0x55b276ff4000 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f098aa00640 1 -- 172.21.15.3:0/250534488 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f098c031890 con 0x55b276ff4000 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.311+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c041070 con 0x55b276ff4000 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f098aa00640 1 -- 172.21.15.3:0/250534488 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f098c068050 con 0x55b276ff4000 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 10 client.26383 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b277079450 con 0x55b276ff4000 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f098aa00640 1 -- 172.21.15.3:0/250534488 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098c041070 con 0x55b276ff4000 2024-08-01T04:26:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f098aa00640 1 -- 172.21.15.3:0/250534488 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f098c004ba0 con 0x55b276ff4000 2024-08-01T04:26:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 10 client.26383 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 10 client.26383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:26:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 20 client.26383 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b277079ae0 0x55b27707bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 4 client.26383 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:26:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.312+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b276f60670 con 0x55b277079ae0 2024-08-01T04:26:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.313+0000 7f09a5800640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b277079ae0 0x55b27707bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.313+0000 7f09a5800640 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b277079ae0 0x55b27707bf00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aefdd8447b0e51fd server_cookie=f07fe114f52897e6 in_seq=0 out_seq=0 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.313+0000 7f098aa00640 10 client.26383 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f098aa00640 1 -- 172.21.15.3:0/250534488 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b276f60670 con 0x55b277079ae0 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f098aa00640 10 client.26383 handle_command_reply: tid=0 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 client.26383 shutdown 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 msgr2=0x55b276ff8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ff8c30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f098c031f30 tx=0x7f098c003c10 comp rx=0 tx=0).stop 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 shutdown_connections 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b276ffe8c0 0x55b276ff9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55b277079ae0 0x55b27707bf00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b276ff4000 0x55b276ff8c30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 --2- 172.21.15.3:0/250534488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b276ff36e0 0x55b276ff86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f098aa00640 0 client.26383 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 >> 172.21.15.3:0/250534488 conn(0x55b276fefc70 msgr2=0x55b276feeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 shutdown_connections 2024-08-01T04:26:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 1 -- 172.21.15.3:0/250534488 wait complete. 2024-08-01T04:26:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.314+0000 7f09aa473740 20 client.26383 trim_cache size 0 max 16384 2024-08-01T04:26:22.324 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:26:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f09840a2cf0 msgr2=0x7f09840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f09840a2cf0 0x7f09840a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55b276ff0ad0 tx=0x7f09900d5530 comp rx=0 tx=0).stop 2024-08-01T04:26:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 msgr2=0x7f09a019a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 0x7f09a019a1d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f099000e9f0 tx=0x7f099000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 shutdown_connections 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f09840a2cf0 0x7f09840a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09a01076f0 0x7f09a019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f09a0102090 0x7f09a019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 --2- 172.21.15.3:0/2854346552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09a01016f0 0x7f09a019a1d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 >> 172.21.15.3:0/2854346552 conn(0x7f09a00fd470 msgr2=0x7f09a010ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 shutdown_connections 2024-08-01T04:26:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:22.315+0000 7f09a6200640 1 -- 172.21.15.3:0/2854346552 wait complete. 2024-08-01T04:26:23.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:26:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:24.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:23 smithi163 ceph-mon[37383]: pgmap v314: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:24.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:23 smithi003 ceph-mon[32262]: pgmap v314: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:24.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:23 smithi160 ceph-mon[37224]: pgmap v314: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:24.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.413+0000 7f7f1cc00640 1 -- 172.21.15.3:0/2092837940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f181038e0 msgr2=0x7f7f18103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.413+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/2092837940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f181038e0 0x7f7f18103d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009930 tx=0x7f7f0002a420 comp rx=0 tx=0).stop 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 -- 172.21.15.3:0/2092837940 shutdown_connections 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/2092837940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f18104280 0x7f7f181107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/2092837940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f181038e0 0x7f7f18103d40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/2092837940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 0x7f7f18109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 -- 172.21.15.3:0/2092837940 >> 172.21.15.3:0/2092837940 conn(0x7f7f180ff5a0 msgr2=0x7f7f181019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 -- 172.21.15.3:0/2092837940 shutdown_connections 2024-08-01T04:26:24.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.414+0000 7f7f1cc00640 1 -- 172.21.15.3:0/2092837940 wait complete. 2024-08-01T04:26:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.416+0000 7f7f1cc00640 1 Processor -- start 2024-08-01T04:26:24.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.416+0000 7f7f1cc00640 1 -- start start 2024-08-01T04:26:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f1cc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 0x7f7f1819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f1cc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f18104280 0x7f7f1819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f1cc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 0x7f7f1819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f1cc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18113d30 con 0x7f7f18104280 2024-08-01T04:26:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f1cc00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18113bb0 con 0x7f7f181038e0 2024-08-01T04:26:24.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f1cc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18113eb0 con 0x7f7f181098e0 2024-08-01T04:26:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f17400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 0x7f7f1819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f18104280 0x7f7f1819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 0x7f7f1819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f17400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 0x7f7f1819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50650/0 (socket says 172.21.15.3:50650) 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.417+0000 7f7f17400640 1 -- 172.21.15.3:0/1383046205 learned_addr learned my addr 172.21.15.3:0/1383046205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17e00640 1 -- 172.21.15.3:0/1383046205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 msgr2=0x7f7f1819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17e00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 0x7f7f1819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17e00640 1 -- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f18104280 msgr2=0x7f7f1819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17e00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f18104280 0x7f7f1819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17e00640 1 -- 172.21.15.3:0/1383046205 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1819f1e0 con 0x7f7f181098e0 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17400640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 0x7f7f1819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f17e00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 0x7f7f1819eb00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009fd0 tx=0x7f7f1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f14c00640 1 -- 172.21.15.3:0/1383046205 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000ce20 con 0x7f7f181098e0 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1819f4d0 con 0x7f7f181098e0 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f14c00640 1 -- 172.21.15.3:0/1383046205 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f10005020 con 0x7f7f181098e0 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.418+0000 7f7f14c00640 1 -- 172.21.15.3:0/1383046205 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10010720 con 0x7f7f181098e0 2024-08-01T04:26:24.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.419+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181a6db0 con 0x7f7f181098e0 2024-08-01T04:26:24.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.423+0000 7f7f20cb6740 1 Processor -- start 2024-08-01T04:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.423+0000 7f7f14c00640 1 -- 172.21.15.3:0/1383046205 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f7f10005540 con 0x7f7f181098e0 2024-08-01T04:26:24.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.423+0000 7f7f20cb6740 1 -- start start 2024-08-01T04:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61917ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 0x559b61922380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619228c0 0x559b61924cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559b6187cbf0 con 0x559b619228c0 2024-08-01T04:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x559b61814b20 con 0x559b619176e0 2024-08-01T04:26:24.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559b61878bb0 con 0x559b61918000 2024-08-01T04:26:24.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f17400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 0x559b61922380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61917ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f17400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 0x559b61922380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43420/0 (socket says 172.21.15.3:43420) 2024-08-01T04:26:24.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f17400640 1 -- 172.21.15.3:0/3626048284 learned_addr learned my addr 172.21.15.3:0/3626048284 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:24.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.424+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619228c0 0x559b61924cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f17400640 1 -- 172.21.15.3:0/3626048284 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 msgr2=0x559b61917ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f17400640 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61917ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f17400640 1 -- 172.21.15.3:0/3626048284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619228c0 msgr2=0x559b61924cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f17400640 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619228c0 0x559b61924cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f17400640 1 -- 172.21.15.3:0/3626048284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b6181b7f0 con 0x559b61918000 2024-08-01T04:26:24.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619228c0 0x559b61924cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:24.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f17400640 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 0x559b61922380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f080115c0 tx=0x7f7f08011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1b3ee7671154ef server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:24.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f14c00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7efc0a2cf0 0x7f7efc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f14c00640 1 -- 172.21.15.3:0/1383046205 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f7f100cfc20 con 0x7f7f181098e0 2024-08-01T04:26:24.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.425+0000 7f7f07e00640 1 -- 172.21.15.3:0/3626048284 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800f660 con 0x559b61918000 2024-08-01T04:26:24.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.426+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7efc0a2cf0 0x7f7efc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.426+0000 7f7f07e00640 1 -- 172.21.15.3:0/3626048284 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0800fce0 con 0x559b61918000 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.426+0000 7f7f07e00640 1 -- 172.21.15.3:0/3626048284 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08019d70 con 0x559b61918000 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.426+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3626048284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 msgr2=0x559b61922380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.426+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7efc0a2cf0 0x7f7efc0a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f00006010 tx=0x7f7f0002a420 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 0x559b61922380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7f080115c0 tx=0x7f7f08011a80 comp rx=0 tx=0).stop 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3626048284 shutdown_connections 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619228c0 0x559b61924cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b61918000 0x559b61922380 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3626048284 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61917ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3626048284 >> 172.21.15.3:0/3626048284 conn(0x559b61913c70 msgr2=0x559b61914050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:24.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3626048284 shutdown_connections 2024-08-01T04:26:24.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3626048284 wait complete. 2024-08-01T04:26:24.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.427+0000 7f7f20cb6740 1 -- start start 2024-08-01T04:26:24.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.428+0000 7f7f20cb6740 1 Processor -- start 2024-08-01T04:26:24.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 0x559b6191f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b61918000 0x559b6191fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b61920170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559b61925490 con 0x559b619176e0 2024-08-01T04:26:24.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x559b61837cf0 con 0x559b61918000 2024-08-01T04:26:24.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559b61925610 con 0x559b619228c0 2024-08-01T04:26:24.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f20cb6740 10 client.0 _ll_register_callbacks cb 0x559b61823e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:24.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f17400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 0x559b6191f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f17e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b61920170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f17400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 0x559b6191f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50844/0 (socket says 172.21.15.3:50844) 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f17400640 1 -- 172.21.15.3:0/3516139606 learned_addr learned my addr 172.21.15.3:0/3516139606 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b61918000 0x559b6191fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.429+0000 7f7f17e00640 1 -- 172.21.15.3:0/3516139606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b61918000 msgr2=0x559b6191fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f17e00640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b61918000 0x559b6191fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f17e00640 1 -- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 msgr2=0x559b6191f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f17e00640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 0x559b6191f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f17e00640 1 -- 172.21.15.3:0/3516139606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b615ee3c0 con 0x559b619228c0 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f17400640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 0x559b6191f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b61918000 0x559b6191fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f17e00640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b61920170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f100cd2a0 tx=0x7f7f1009c070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de90daf4763b0e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f04c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f04c00640 1 -- 172.21.15.3:0/3516139606 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100a00d0 con 0x559b619228c0 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f04c00640 1 -- 172.21.15.3:0/3516139606 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f7f100986e0 con 0x559b619228c0 2024-08-01T04:26:24.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f0800fce0 con 0x559b619228c0 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.430+0000 7f7f04c00640 1 -- 172.21.15.3:0/3516139606 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f1009c2d0 con 0x559b619228c0 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 10 client.26395 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b619310f0 con 0x559b619228c0 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f04c00640 1 -- 172.21.15.3:0/3516139606 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0800fce0 con 0x559b619228c0 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f04c00640 1 -- 172.21.15.3:0/3516139606 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7f1009b550 con 0x559b619228c0 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 10 client.26395 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 10 client.26395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:26:24.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 20 client.26395 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b61932500 0x559b61934910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 4 client.26395 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.431+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b6188a160 con 0x559b61932500 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.432+0000 7f7f17400640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b61932500 0x559b61934910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.432+0000 7f7f17400640 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b61932500 0x559b61934910 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11be675b0fedc528 server_cookie=603ec163ef1c82e in_seq=0 out_seq=0 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.432+0000 7f7f04c00640 10 client.26395 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:24.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f04c00640 1 -- 172.21.15.3:0/3516139606 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b6188a160 con 0x559b61932500 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f04c00640 10 client.26395 handle_command_reply: tid=0 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 client.26395 shutdown 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 msgr2=0x559b61920170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b61920170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f100cd2a0 tx=0x7f7f1009c070 comp rx=0 tx=0).stop 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 shutdown_connections 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b61932500 0x559b61934910 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b61920170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b61918000 0x559b6191fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.433+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/3516139606 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b619176e0 0x559b6191f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.434+0000 7f7f04c00640 0 client.26395 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.434+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 >> 172.21.15.3:0/3516139606 conn(0x559b61913c70 msgr2=0x559b61919950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.434+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 shutdown_connections 2024-08-01T04:26:24.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.434+0000 7f7f20cb6740 1 -- 172.21.15.3:0/3516139606 wait complete. 2024-08-01T04:26:24.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.434+0000 7f7f20cb6740 20 client.26395 trim_cache size 0 max 16384 2024-08-01T04:26:24.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.448+0000 7f7f20cb6740 1 Processor -- start 2024-08-01T04:26:24.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.448+0000 7f7f20cb6740 1 -- start start 2024-08-01T04:26:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619176e0 0x559b6191a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b6193a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619228c0 0x559b6193a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181b2110 con 0x559b61918000 2024-08-01T04:26:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181af9e0 con 0x559b619228c0 2024-08-01T04:26:24.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181ad850 con 0x559b619176e0 2024-08-01T04:26:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b6193a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b6193a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50852/0 (socket says 172.21.15.3:50852) 2024-08-01T04:26:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f0fe00640 1 -- 172.21.15.3:0/2831014741 learned_addr learned my addr 172.21.15.3:0/2831014741 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f17e00640 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619228c0 0x559b6193a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.449+0000 7f7f17400640 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619176e0 0x559b6191a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f0fe00640 1 -- 172.21.15.3:0/2831014741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619176e0 msgr2=0x559b6191a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619176e0 0x559b6191a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f0fe00640 1 -- 172.21.15.3:0/2831014741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619228c0 msgr2=0x559b6193a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619228c0 0x559b6193a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f0fe00640 1 -- 172.21.15.3:0/2831014741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b61917e30 con 0x559b61918000 2024-08-01T04:26:24.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f17e00640 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619228c0 0x559b6193a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f0fe00640 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b6193a1b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7f0002aa10 tx=0x7f7f0003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b30f6292a1f2483 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f07400640 1 -- 172.21.15.3:0/2831014741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00004980 con 0x559b61918000 2024-08-01T04:26:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f07400640 1 -- 172.21.15.3:0/2831014741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0004f070 con 0x559b61918000 2024-08-01T04:26:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.450+0000 7f7f07400640 1 -- 172.21.15.3:0/2831014741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00003850 con 0x559b61918000 2024-08-01T04:26:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 -- 172.21.15.3:0/2831014741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 msgr2=0x559b6193a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b6193a1b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7f0002aa10 tx=0x7f7f0003f040 comp rx=0 tx=0).stop 2024-08-01T04:26:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 -- 172.21.15.3:0/2831014741 shutdown_connections 2024-08-01T04:26:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619228c0 0x559b6193a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b6193a1b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/2831014741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619176e0 0x559b6191a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 -- 172.21.15.3:0/2831014741 >> 172.21.15.3:0/2831014741 conn(0x559b61913c70 msgr2=0x559b6191e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:24.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 -- 172.21.15.3:0/2831014741 shutdown_connections 2024-08-01T04:26:24.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.451+0000 7f7f20cb6740 1 -- 172.21.15.3:0/2831014741 wait complete. 2024-08-01T04:26:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 -- start start 2024-08-01T04:26:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 Processor -- start 2024-08-01T04:26:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61925c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b61926140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b6199b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181adea0 con 0x559b61918000 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181af310 con 0x559b619176e0 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.452+0000 7f7f20cb6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181ae570 con 0x559b619228c0 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f20cb6740 10 client.0 _ll_register_callbacks cb 0x7f7f1009ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b61926140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f0fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b61926140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50862/0 (socket says 172.21.15.3:50862) 2024-08-01T04:26:24.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f0fe00640 1 -- 172.21.15.3:0/661479484 learned_addr learned my addr 172.21.15.3:0/661479484 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17e00640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b6199b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17400640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61925c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17e00640 1 -- 172.21.15.3:0/661479484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 msgr2=0x559b61925c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17e00640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61925c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17e00640 1 -- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 msgr2=0x559b61926140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17e00640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b61926140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.453+0000 7f7f17e00640 1 -- 172.21.15.3:0/661479484 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6199b6b0 con 0x559b619228c0 2024-08-01T04:26:24.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f17e00640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b6199b170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f100a47b0 tx=0x7f7f10005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ec1a9ab3dd684e server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f04c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f04c00640 1 -- 172.21.15.3:0/661479484 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1009be40 con 0x559b619228c0 2024-08-01T04:26:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f04c00640 1 -- 172.21.15.3:0/661479484 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f7f100986e0 con 0x559b619228c0 2024-08-01T04:26:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f0004f070 con 0x559b619228c0 2024-08-01T04:26:24.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f04c00640 1 -- 172.21.15.3:0/661479484 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f100a4850 con 0x559b619228c0 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f20cb6740 10 client.26401 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b6199c8b0 con 0x559b619228c0 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.454+0000 7f7f04c00640 1 -- 172.21.15.3:0/661479484 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f0004f070 con 0x559b619228c0 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f04c00640 1 -- 172.21.15.3:0/661479484 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7f10085080 con 0x559b619228c0 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f20cb6740 10 client.26401 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f20cb6740 10 client.26401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f20cb6740 20 client.26401 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b6199d080 0x559b6199f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:24.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f20cb6740 4 client.26401 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b61884670 con 0x559b6199d080 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.455+0000 7f7f17400640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b6199d080 0x559b6199f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f17400640 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b6199d080 0x559b6199f4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8296b7c1df9b73ed server_cookie=74934fdb1473e620 in_seq=0 out_seq=0 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f04c00640 10 client.26401 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f04c00640 1 -- 172.21.15.3:0/661479484 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b61884670 con 0x559b6199d080 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f04c00640 10 client.26401 handle_command_reply: tid=0 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 client.26401 shutdown 2024-08-01T04:26:24.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 msgr2=0x559b6199b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b6199b170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f100a47b0 tx=0x7f7f10005940 comp rx=0 tx=0).stop 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 shutdown_connections 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x559b6199d080 0x559b6199f4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559b619228c0 0x559b6199b170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559b61918000 0x559b61926140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 --2- 172.21.15.3:0/661479484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559b619176e0 0x559b61925c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 >> 172.21.15.3:0/661479484 conn(0x559b61913c70 msgr2=0x559b61912480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f04c00640 0 client.26401 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 shutdown_connections 2024-08-01T04:26:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.456+0000 7f7f20cb6740 1 -- 172.21.15.3:0/661479484 wait complete. 2024-08-01T04:26:24.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f20cb6740 20 client.26401 trim_cache size 0 max 16384 2024-08-01T04:26:24.467 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:26:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7efc0a2cf0 msgr2=0x7f7efc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7efc0a2cf0 0x7f7efc0a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f00006010 tx=0x7f7f0002a420 comp rx=0 tx=0).stop 2024-08-01T04:26:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 msgr2=0x7f7f1819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 0x7f7f1819eb00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009fd0 tx=0x7f7f1000ef10 comp rx=0 tx=0).stop 2024-08-01T04:26:24.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 shutdown_connections 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7efc0a2cf0 0x7f7efc0a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f181098e0 0x7f7f1819eb00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f18104280 0x7f7f1819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 --2- 172.21.15.3:0/1383046205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f181038e0 0x7f7f1819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.457+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 >> 172.21.15.3:0/1383046205 conn(0x7f7f180ff5a0 msgr2=0x7f7f180ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.458+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 shutdown_connections 2024-08-01T04:26:24.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:24.458+0000 7f7f1cc00640 1 -- 172.21.15.3:0/1383046205 wait complete. 2024-08-01T04:26:26.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:25 smithi163 ceph-mon[37383]: pgmap v315: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:26.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:25 smithi003 ceph-mon[32262]: pgmap v315: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:26.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:25 smithi160 ceph-mon[37224]: pgmap v315: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:26:26.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:26:27.485 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:27.901 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:27 smithi003 ceph-mon[32262]: pgmap v316: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:26:28.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:27 smithi163 ceph-mon[37383]: pgmap v316: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:26:28.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:27 smithi160 ceph-mon[37224]: pgmap v316: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:26:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 -- 172.21.15.3:0/4183388304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258010dd90 msgr2=0x7f258010e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 --2- 172.21.15.3:0/4183388304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258010dd90 0x7f258010e170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2570009cd0 tx=0x7f257002a8f0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 -- 172.21.15.3:0/4183388304 shutdown_connections 2024-08-01T04:26:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 --2- 172.21.15.3:0/4183388304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25801086b0 0x7f2580114bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 --2- 172.21.15.3:0/4183388304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2580107d50 0x7f2580108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 --2- 172.21.15.3:0/4183388304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258010dd90 0x7f258010e170 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.290+0000 7f257f200640 1 -- 172.21.15.3:0/4183388304 >> 172.21.15.3:0/4183388304 conn(0x7f25800776b0 msgr2=0x7f2580079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.291+0000 7f257f200640 1 -- 172.21.15.3:0/4183388304 shutdown_connections 2024-08-01T04:26:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.291+0000 7f257f200640 1 -- 172.21.15.3:0/4183388304 wait complete. 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.292+0000 7f257f200640 1 Processor -- start 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.292+0000 7f257f200640 1 -- start start 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257f200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 0x7f258019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257f200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25801086b0 0x7f258019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258010dd90 0x7f25801a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257f200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25801180e0 con 0x7f25801086b0 2024-08-01T04:26:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2580117f60 con 0x7f258010dd90 2024-08-01T04:26:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257f200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f2580118260 con 0x7f2580107d50 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 0x7f258019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257de00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 0x7f258019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43448/0 (socket says 172.21.15.3:43448) 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257de00640 1 -- 172.21.15.3:0/474955641 learned_addr learned my addr 172.21.15.3:0/474955641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f2575400640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25801086b0 0x7f258019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.293+0000 7f257e800640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258010dd90 0x7f25801a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.294+0000 7f257de00640 1 -- 172.21.15.3:0/474955641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258010dd90 msgr2=0x7f25801a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.294+0000 7f257de00640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258010dd90 0x7f25801a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.294+0000 7f257de00640 1 -- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25801086b0 msgr2=0x7f258019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.294+0000 7f257de00640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25801086b0 0x7f258019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.294+0000 7f257de00640 1 -- 172.21.15.3:0/474955641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25801a3550 con 0x7f2580107d50 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.294+0000 7f257de00640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 0x7f258019e5a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2570000f80 tx=0x7f2570003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.295+0000 7f2577400640 1 -- 172.21.15.3:0/474955641 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570030e40 con 0x7f2580107d50 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.295+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25801a37e0 con 0x7f2580107d50 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.295+0000 7f2577400640 1 -- 172.21.15.3:0/474955641 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2570040070 con 0x7f2580107d50 2024-08-01T04:26:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.295+0000 7f2577400640 1 -- 172.21.15.3:0/474955641 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570045ae0 con 0x7f2580107d50 2024-08-01T04:26:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.295+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801ab120 con 0x7f2580107d50 2024-08-01T04:26:28.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 Processor -- start 2024-08-01T04:26:28.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 -- start start 2024-08-01T04:26:28.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2577400640 1 -- 172.21.15.3:0/474955641 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2570004040 con 0x7f2580107d50 2024-08-01T04:26:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 0x55b6be9d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b6be93dbf0 con 0x55b6be9d86e0 2024-08-01T04:26:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b6be8d5b20 con 0x55b6be9d9000 2024-08-01T04:26:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.300+0000 7f2587bc3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b6be939bb0 con 0x55b6be9e38c0 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 0x55b6be9d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 0x55b6be9d8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50894/0 (socket says 172.21.15.3:50894) 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 -- 172.21.15.3:0/50840600 learned_addr learned my addr 172.21.15.3:0/50840600 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257de00640 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f2575400640 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 -- 172.21.15.3:0/50840600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 msgr2=0x55b6be9e5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 -- 172.21.15.3:0/50840600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 msgr2=0x55b6be9e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f257e800640 1 -- 172.21.15.3:0/50840600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6be8dc7f0 con 0x55b6be9d86e0 2024-08-01T04:26:28.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.301+0000 7f2575400640 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:28.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f257e800640 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 0x55b6be9d8ac0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2578009c10 tx=0x7f2578011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72e063f0082d44c server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f256f400640 1 -- 172.21.15.3:0/50840600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257801e070 con 0x55b6be9d86e0 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f256f400640 1 -- 172.21.15.3:0/50840600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578019b10 con 0x55b6be9d86e0 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f256f400640 1 -- 172.21.15.3:0/50840600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257800e9d0 con 0x55b6be9d86e0 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f2577400640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f25640a2d70 0x7f25640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f2577400640 1 -- 172.21.15.3:0/474955641 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f25700f03f0 con 0x7f2580107d50 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.302+0000 7f257de00640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f25640a2d70 0x7f25640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 -- 172.21.15.3:0/50840600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 msgr2=0x55b6be9d8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 0x55b6be9d8ac0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2578009c10 tx=0x7f2578011810 comp rx=0 tx=0).stop 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 -- 172.21.15.3:0/50840600 shutdown_connections 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 --2- 172.21.15.3:0/50840600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d86e0 0x55b6be9d8ac0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 -- 172.21.15.3:0/50840600 >> 172.21.15.3:0/50840600 conn(0x55b6be9d4c70 msgr2=0x55b6be9d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 -- 172.21.15.3:0/50840600 shutdown_connections 2024-08-01T04:26:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f2587bc3740 1 -- 172.21.15.3:0/50840600 wait complete. 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.303+0000 7f257de00640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f25640a2d70 0x7f25640a5230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25700f3fd0 tx=0x7f25700f3ea0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.304+0000 7f2587bc3740 1 -- start start 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.304+0000 7f2587bc3740 1 Processor -- start 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b6be9e6490 con 0x55b6be9d9000 2024-08-01T04:26:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55b6be8f8cf0 con 0x55b6be9d86e0 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55b6be9e6610 con 0x55b6be9e38c0 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2587bc3740 10 client.0 _ll_register_callbacks cb 0x55b6be8e4e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2575400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2575400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9e0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50902/0 (socket says 172.21.15.3:50902) 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f257e800640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9e0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f2575400640 1 -- 172.21.15.3:0/832226345 learned_addr learned my addr 172.21.15.3:0/832226345 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.305+0000 7f257e800640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2575400640 1 -- 172.21.15.3:0/832226345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 msgr2=0x55b6be9e1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2575400640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2575400640 1 -- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 msgr2=0x55b6be9e0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2575400640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9e0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2575400640 1 -- 172.21.15.3:0/832226345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6be6af3c0 con 0x55b6be9d9000 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f257e800640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2575400640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9e0d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2568012700 tx=0x7f2568012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396aad622f90442b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:28.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2563e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:26:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2563e00640 1 -- 172.21.15.3:0/832226345 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568004030 con 0x55b6be9d9000 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2563e00640 1 -- 172.21.15.3:0/832226345 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f25680041d0 con 0x55b6be9d9000 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.306+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2578019b10 con 0x55b6be9d9000 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2563e00640 1 -- 172.21.15.3:0/832226345 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f25680043d0 con 0x55b6be9d9000 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6be9f2200 con 0x55b6be9d9000 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2563e00640 1 -- 172.21.15.3:0/832226345 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2578019b10 con 0x55b6be9d9000 2024-08-01T04:26:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2563e00640 1 -- 172.21.15.3:0/832226345 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f2568003c50 con 0x55b6be9d9000 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 20 client.16254 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6be9f36c0 0x55b6be9f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 4 client.16254 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.307+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6be94b160 con 0x55b6be9f36c0 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.308+0000 7f257e800640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6be9f36c0 0x55b6be9f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.308+0000 7f257e800640 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6be9f36c0 0x55b6be9f5af0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0d7471aa7ccafd server_cookie=5dc4d2ce52c601f9 in_seq=0 out_seq=0 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.308+0000 7f2563e00640 10 client.16254 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.309+0000 7f2563e00640 1 -- 172.21.15.3:0/832226345 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b6be94b160 con 0x55b6be9f36c0 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.309+0000 7f2563e00640 10 client.16254 handle_command_reply: tid=0 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.309+0000 7f2587bc3740 1 client.16254 shutdown 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 msgr2=0x55b6be9e0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9e0d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2568012700 tx=0x7f2568012bc0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 shutdown_connections 2024-08-01T04:26:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6be9f36c0 0x55b6be9f5af0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9e0d50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 --2- 172.21.15.3:0/832226345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2563e00640 0 client.16254 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 >> 172.21.15.3:0/832226345 conn(0x55b6be9d4c70 msgr2=0x55b6be9da9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 shutdown_connections 2024-08-01T04:26:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 1 -- 172.21.15.3:0/832226345 wait complete. 2024-08-01T04:26:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.310+0000 7f2587bc3740 20 client.16254 trim_cache size 0 max 16384 2024-08-01T04:26:28.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 Processor -- start 2024-08-01T04:26:28.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 -- start start 2024-08-01T04:26:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9d86e0 0x55b6be9db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 0x55b6be9fb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b6460 con 0x55b6be9e38c0 2024-08-01T04:26:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b3d30 con 0x55b6be9d9000 2024-08-01T04:26:28.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f2587bc3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b1ba0 con 0x55b6be9d86e0 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.325+0000 7f257e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 0x55b6be9fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 0x55b6be9fb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50916/0 (socket says 172.21.15.3:50916) 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f2575400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 -- 172.21.15.3:0/469955424 learned_addr learned my addr 172.21.15.3:0/469955424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9d86e0 0x55b6be9db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 -- 172.21.15.3:0/469955424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9d86e0 msgr2=0x55b6be9db140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9d86e0 0x55b6be9db140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 -- 172.21.15.3:0/469955424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 msgr2=0x55b6be9e2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 -- 172.21.15.3:0/469955424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6be9d5d20 con 0x55b6be9e38c0 2024-08-01T04:26:28.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.326+0000 7f257e800640 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 0x55b6be9fb400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2578009be0 tx=0x7f2578005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3060481d435930 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:28.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.327+0000 7f256ea00640 1 -- 172.21.15.3:0/469955424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578035070 con 0x55b6be9e38c0 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.327+0000 7f256ea00640 1 -- 172.21.15.3:0/469955424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257800e660 con 0x55b6be9e38c0 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.327+0000 7f256ea00640 1 -- 172.21.15.3:0/469955424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257801e040 con 0x55b6be9e38c0 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 -- 172.21.15.3:0/469955424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 msgr2=0x55b6be9fb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 0x55b6be9fb400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2578009be0 tx=0x7f2578005f90 comp rx=0 tx=0).stop 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 -- 172.21.15.3:0/469955424 shutdown_connections 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9e38c0 0x55b6be9fb400 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d9000 0x55b6be9e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 --2- 172.21.15.3:0/469955424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9d86e0 0x55b6be9db140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 -- 172.21.15.3:0/469955424 >> 172.21.15.3:0/469955424 conn(0x55b6be9d4c70 msgr2=0x55b6be9e2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 -- 172.21.15.3:0/469955424 shutdown_connections 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 -- 172.21.15.3:0/469955424 wait complete. 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 -- start start 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.328+0000 7f2587bc3740 1 Processor -- start 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b21f0 con 0x55b6be9d9000 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b3660 con 0x55b6be9d86e0 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f25801b28c0 con 0x55b6be9e38c0 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2587bc3740 10 client.0 _ll_register_callbacks cb 0x7f2568009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2575400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2575400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50926/0 (socket says 172.21.15.3:50926) 2024-08-01T04:26:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f257e800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f2575400640 1 -- 172.21.15.3:0/2271206221 learned_addr learned my addr 172.21.15.3:0/2271206221 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.329+0000 7f257e800640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2575400640 1 -- 172.21.15.3:0/2271206221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 msgr2=0x55b6be9dcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2575400640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9dcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2575400640 1 -- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 msgr2=0x55b6be9eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2575400640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2575400640 1 -- 172.21.15.3:0/2271206221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6be9dd1a0 con 0x55b6be9d9000 2024-08-01T04:26:28.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f257e800640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9dcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2575400640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9ebda0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55b6be9e1e50 tx=0x7f2568010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8633e7bdda994a7a server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.330+0000 7f2563e00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:26:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2563e00640 1 -- 172.21.15.3:0/2271206221 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568031060 con 0x55b6be9d9000 2024-08-01T04:26:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2563e00640 1 -- 172.21.15.3:0/2271206221 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f2568010cf0 con 0x55b6be9d9000 2024-08-01T04:26:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f257800e660 con 0x55b6be9d9000 2024-08-01T04:26:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2563e00640 1 -- 172.21.15.3:0/2271206221 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f256800f790 con 0x55b6be9d9000 2024-08-01T04:26:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2587bc3740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6bea5dae0 con 0x55b6be9d9000 2024-08-01T04:26:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2563e00640 1 -- 172.21.15.3:0/2271206221 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f257800e660 con 0x55b6be9d9000 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2563e00640 1 -- 172.21.15.3:0/2271206221 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f256801ca90 con 0x55b6be9d9000 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2587bc3740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.331+0000 7f2587bc3740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.332+0000 7f2587bc3740 20 client.16260 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.332+0000 7f2587bc3740 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6bea5e1e0 0x55b6bea60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.332+0000 7f2587bc3740 4 client.16260 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.332+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6be945670 con 0x55b6bea5e1e0 2024-08-01T04:26:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.332+0000 7f257e800640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6bea5e1e0 0x55b6bea60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f257e800640 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6bea5e1e0 0x55b6bea60600 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638ed7f35fcd739b server_cookie=e36074d715af8684 in_seq=0 out_seq=0 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2563e00640 10 client.16260 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2563e00640 1 -- 172.21.15.3:0/2271206221 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6be945670 con 0x55b6bea5e1e0 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2563e00640 10 client.16260 handle_command_reply: tid=0 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 client.16260 shutdown 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 msgr2=0x55b6be9ebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9ebda0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55b6be9e1e50 tx=0x7f2568010870 comp rx=0 tx=0).stop 2024-08-01T04:26:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 shutdown_connections 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55b6bea5e1e0 0x55b6bea60600 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55b6be9e38c0 0x55b6be9dcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b6be9d9000 0x55b6be9ebda0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 --2- 172.21.15.3:0/2271206221 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b6be9d86e0 0x55b6be9eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2563e00640 0 client.16260 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.333+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 >> 172.21.15.3:0/2271206221 conn(0x55b6be9d4c70 msgr2=0x55b6be9e2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 shutdown_connections 2024-08-01T04:26:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f2587bc3740 1 -- 172.21.15.3:0/2271206221 wait complete. 2024-08-01T04:26:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f2587bc3740 20 client.16260 trim_cache size 0 max 16384 2024-08-01T04:26:28.343 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:26:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f25640a2d70 msgr2=0x7f25640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f257f200640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f25640a2d70 0x7f25640a5230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25700f3fd0 tx=0x7f25700f3ea0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 msgr2=0x7f258019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.334+0000 7f257f200640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 0x7f258019e5a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2570000f80 tx=0x7f2570003ea0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 shutdown_connections 2024-08-01T04:26:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258010dd90 0x7f25801a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25801086b0 0x7f258019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f25640a2d70 0x7f25640a5230 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 --2- 172.21.15.3:0/474955641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2580107d50 0x7f258019e5a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 >> 172.21.15.3:0/474955641 conn(0x7f25800776b0 msgr2=0x7f2580078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 shutdown_connections 2024-08-01T04:26:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:28.335+0000 7f257f200640 1 -- 172.21.15.3:0/474955641 wait complete. 2024-08-01T04:26:30.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:29 smithi163 ceph-mon[37383]: pgmap v317: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:26:30.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:29 smithi003 ceph-mon[32262]: pgmap v317: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:26:30.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:29 smithi160 ceph-mon[37224]: pgmap v317: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:26:30.169 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-01T04:26:30.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-01T04:26:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.325+0000 7f7ee1600640 1 -- 172.21.15.3:0/295827570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100fb0 msgr2=0x7f7edc10dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.325+0000 7f7ee1600640 1 --2- 172.21.15.3:0/295827570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100fb0 0x7f7edc10dc70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009900 tx=0x7f7ec802a3f0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 -- 172.21.15.3:0/295827570 shutdown_connections 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 --2- 172.21.15.3:0/295827570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc10e1b0 0x7f7edc1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 --2- 172.21.15.3:0/295827570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100fb0 0x7f7edc10dc70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 --2- 172.21.15.3:0/295827570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7edc100600 0x7f7edc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 -- 172.21.15.3:0/295827570 >> 172.21.15.3:0/295827570 conn(0x7f7edc0fc4d0 msgr2=0x7f7edc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 -- 172.21.15.3:0/295827570 shutdown_connections 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.326+0000 7f7ee1600640 1 -- 172.21.15.3:0/295827570 wait complete. 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.327+0000 7f7ee1600640 1 Processor -- start 2024-08-01T04:26:31.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.327+0000 7f7ee1600640 1 -- start start 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7ee1600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 0x7f7edc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7ee1600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc100fb0 0x7f7edc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7ee1600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7edc10e1b0 0x7f7edc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7ee1600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc113f20 con 0x7f7edc100fb0 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7ee1600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc113da0 con 0x7f7edc10e1b0 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7ee1600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1140a0 con 0x7f7edc100600 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 0x7f7edc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 0x7f7edc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43506/0 (socket says 172.21.15.3:43506) 2024-08-01T04:26:31.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7edaa00640 1 -- 172.21.15.3:0/2777299849 learned_addr learned my addr 172.21.15.3:0/2777299849 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.328+0000 7f7eda000640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc100fb0 0x7f7edc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7edaa00640 1 -- 172.21.15.3:0/2777299849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7edc10e1b0 msgr2=0x7f7edc19ed10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7edaa00640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7edc10e1b0 0x7f7edc19ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7edaa00640 1 -- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc100fb0 msgr2=0x7f7edc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7edaa00640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc100fb0 0x7f7edc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7edaa00640 1 -- 172.21.15.3:0/2777299849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc19f3f0 con 0x7f7edc100600 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7eda000640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc100fb0 0x7f7edc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:31.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.329+0000 7f7edaa00640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 0x7f7edc19a440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00e9f0 tx=0x7f7ecc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.330+0000 7f7ed3400640 1 -- 172.21.15.3:0/2777299849 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00cd80 con 0x7f7edc100600 2024-08-01T04:26:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.330+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc19f6e0 con 0x7f7edc100600 2024-08-01T04:26:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.330+0000 7f7ed3400640 1 -- 172.21.15.3:0/2777299849 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc005500 con 0x7f7edc100600 2024-08-01T04:26:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.330+0000 7f7ed3400640 1 -- 172.21.15.3:0/2777299849 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc010690 con 0x7f7edc100600 2024-08-01T04:26:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.330+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc1a6fc0 con 0x7f7edc100600 2024-08-01T04:26:31.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.333+0000 7f7ed3400640 1 -- 172.21.15.3:0/2777299849 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f7ecc0046c0 con 0x7f7edc100600 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.335+0000 7f7ed3400640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7ec00a2cf0 0x7f7ec00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.335+0000 7f7ee3fc9740 1 Processor -- start 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.335+0000 7f7ed3400640 1 -- 172.21.15.3:0/2777299849 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f7ecc0cfa00 con 0x7f7edc100600 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.335+0000 7f7ee3fc9740 1 -- start start 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.335+0000 7f7eda000640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7ec00a2cf0 0x7f7ec00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700425860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700430120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700430660 0x55a700432a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a7003407e0 con 0x55a700425da0 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a70038bac0 con 0x55a700425480 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a7003238c0 con 0x55a700430660 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7eda000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700430120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700425860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7eda000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700430120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50948/0 (socket says 172.21.15.3:50948) 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.336+0000 7f7eda000640 1 -- 172.21.15.3:0/2892129578 learned_addr learned my addr 172.21.15.3:0/2892129578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:31.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edb400640 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700430660 0x55a700432a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edaa00640 1 -- 172.21.15.3:0/2892129578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700430660 msgr2=0x55a700432a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7eda000640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7ec00a2cf0 0x7f7ec00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8000c00 tx=0x7f7ec80040c0 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edaa00640 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700430660 0x55a700432a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edaa00640 1 -- 172.21.15.3:0/2892129578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 msgr2=0x55a700430120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edaa00640 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700430120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edaa00640 1 -- 172.21.15.3:0/2892129578 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70032a760 con 0x55a700425480 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7edaa00640 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700425860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0cd080 tx=0x7f7ecc09ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff2c3729630cb14 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:31.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.337+0000 7f7ec7400640 1 -- 172.21.15.3:0/2892129578 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0a0200 con 0x55a700425480 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ec7400640 1 -- 172.21.15.3:0/2892129578 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc09e070 con 0x55a700425480 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ec7400640 1 -- 172.21.15.3:0/2892129578 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0a4630 con 0x55a700425480 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/2892129578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 msgr2=0x55a700425860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700425860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0cd080 tx=0x7f7ecc09ba50 comp rx=0 tx=0).stop 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/2892129578 shutdown_connections 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700430660 0x55a700432a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700430120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/2892129578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700425860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.338+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/2892129578 >> 172.21.15.3:0/2892129578 conn(0x55a700421a10 msgr2=0x55a700421df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.339+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/2892129578 shutdown_connections 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.339+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/2892129578 wait complete. 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.339+0000 7f7ee3fc9740 1 -- start start 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 Processor -- start 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a70042d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425da0 0x55a70042dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700430660 0x55a70042e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a700346ae0 con 0x55a700430660 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a700387a80 con 0x55a700425480 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a700432ff0 con 0x55a700425da0 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.340+0000 7f7ee3fc9740 10 client.0 _ll_register_callbacks cb 0x55a700348190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a70042d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a70042d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50762/0 (socket says 172.21.15.3:50762) 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 -- 172.21.15.3:0/746593131 learned_addr learned my addr 172.21.15.3:0/746593131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edb400640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700430660 0x55a70042e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7eda000640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425da0 0x55a70042dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 -- 172.21.15.3:0/746593131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425da0 msgr2=0x55a70042dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425da0 0x55a70042dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 -- 172.21.15.3:0/746593131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700430660 msgr2=0x55a70042e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700430660 0x55a70042e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 -- 172.21.15.3:0/746593131 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7000fd6a0 con 0x55a700425480 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7edaa00640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a70042d7e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c6a0 tx=0x7f7ecc004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714db94f864cfbf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.341+0000 7f7ebbe00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:26:31.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ebbe00640 1 -- 172.21.15.3:0/746593131 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0a4ac0 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ebbe00640 1 -- 172.21.15.3:0/746593131 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f7ecc09b4b0 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ecc09e070 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ebbe00640 1 -- 172.21.15.3:0/746593131 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ecc0cd470 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ee3fc9740 10 client.26456 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a70043edc0 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ebbe00640 1 -- 172.21.15.3:0/746593131 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ecc09e070 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ebbe00640 1 -- 172.21.15.3:0/746593131 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7ecc0baa90 con 0x55a700425480 2024-08-01T04:26:31.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ee3fc9740 10 client.26456 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ee3fc9740 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.342+0000 7f7ee3fc9740 20 client.26456 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.343+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700440110 0x55a700442500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.343+0000 7f7ee3fc9740 4 client.26456 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.343+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a700399030 con 0x55a700440110 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.343+0000 7f7eda000640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700440110 0x55a700442500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.344+0000 7f7eda000640 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700440110 0x55a700442500 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1131367f276a42 server_cookie=53dd0673d8329627 in_seq=0 out_seq=0 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.344+0000 7f7ebbe00640 10 client.26456 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:31.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.344+0000 7f7ebbe00640 1 -- 172.21.15.3:0/746593131 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a700399030 con 0x55a700440110 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.344+0000 7f7ebbe00640 10 client.26456 handle_command_reply: tid=0 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.344+0000 7f7ee3fc9740 1 client.26456 shutdown 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 msgr2=0x55a70042d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a70042d7e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c6a0 tx=0x7f7ecc004b10 comp rx=0 tx=0).stop 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 shutdown_connections 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700440110 0x55a700442500 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700430660 0x55a70042e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425da0 0x55a70042dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ebbe00640 0 client.26456 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/746593131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a70042d7e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 >> 172.21.15.3:0/746593131 conn(0x55a700421a10 msgr2=0x55a700427740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 shutdown_connections 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/746593131 wait complete. 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.345+0000 7f7ee3fc9740 20 client.26456 trim_cache size 0 max 16384 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.364+0000 7f7ee3fc9740 1 Processor -- start 2024-08-01T04:26:31.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.364+0000 7f7ee3fc9740 1 -- start start 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700421a10 0x55a70042c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 0x55a700445520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425da0 0x55a700445a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1b2350 con 0x55a700421a10 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1afc20 con 0x55a700425da0 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ada90 con 0x55a700425480 2024-08-01T04:26:31.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7eda000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 0x55a700445520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7eda000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 0x55a700445520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43532/0 (socket says 172.21.15.3:43532) 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7eda000640 1 -- 172.21.15.3:0/1068170727 learned_addr learned my addr 172.21.15.3:0/1068170727 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7edaa00640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700421a10 0x55a70042c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.365+0000 7f7edb400640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425da0 0x55a700445a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7eda000640 1 -- 172.21.15.3:0/1068170727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425da0 msgr2=0x55a700445a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7eda000640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425da0 0x55a700445a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7eda000640 1 -- 172.21.15.3:0/1068170727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700421a10 msgr2=0x55a70042c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7eda000640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700421a10 0x55a70042c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7eda000640 1 -- 172.21.15.3:0/1068170727 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a700422ac0 con 0x55a700425480 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7edb400640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425da0 0x55a700445a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7edaa00640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700421a10 0x55a70042c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7eda000640 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 0x55a700445520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009230 tx=0x7f7ec8044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d801e3d1684a361 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7ec6a00640 1 -- 172.21.15.3:0/1068170727 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8035050 con 0x55a700425480 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7ec6a00640 1 -- 172.21.15.3:0/1068170727 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8004ba0 con 0x55a700425480 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.366+0000 7f7ec6a00640 1 -- 172.21.15.3:0/1068170727 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8032db0 con 0x55a700425480 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1068170727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 msgr2=0x55a700445520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 0x55a700445520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009230 tx=0x7f7ec8044000 comp rx=0 tx=0).stop 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1068170727 shutdown_connections 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425da0 0x55a700445a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700425480 0x55a700445520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1068170727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700421a10 0x55a70042c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1068170727 >> 172.21.15.3:0/1068170727 conn(0x55a70042de80 msgr2=0x55a70042c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1068170727 shutdown_connections 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.367+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1068170727 wait complete. 2024-08-01T04:26:31.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 -- start start 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 Processor -- start 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 0x55a700443570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700443ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700443ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ae0e0 con 0x55a700425da0 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1af550 con 0x55a700425480 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ae7b0 con 0x55a700421a10 2024-08-01T04:26:31.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.368+0000 7f7ee3fc9740 10 client.0 _ll_register_callbacks cb 0x7f7ec8041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7eda000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700443ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 0x55a700443570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edb400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700443ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 0x55a700443570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:43542/0 (socket says 172.21.15.3:43542) 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 -- 172.21.15.3:0/1097941161 learned_addr learned my addr 172.21.15.3:0/1097941161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7eda000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700443ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50784/0 (socket says 172.21.15.3:50784) 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 -- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 msgr2=0x55a700443ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700443ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 -- 172.21.15.3:0/1097941161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 msgr2=0x55a700443ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700443ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 -- 172.21.15.3:0/1097941161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7004226a0 con 0x55a700421a10 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7eda000640 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700443ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7edaa00640 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 0x55a700443570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc09a100 tx=0x7f7ecc09b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2b0a5bdebec540 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.369+0000 7f7ebbe00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ebbe00640 1 -- 172.21.15.3:0/1097941161 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc09bc20 con 0x55a700421a10 2024-08-01T04:26:31.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec8004ba0 con 0x55a700421a10 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ebbe00640 1 -- 172.21.15.3:0/1097941161 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f7ecc097db0 con 0x55a700421a10 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ebbe00640 1 -- 172.21.15.3:0/1097941161 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ecc0ab050 con 0x55a700421a10 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 10 client.26437 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7004aae10 con 0x55a700421a10 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ebbe00640 1 -- 172.21.15.3:0/1097941161 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec8004ba0 con 0x55a700421a10 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ebbe00640 1 -- 172.21.15.3:0/1097941161 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f7ecc004f20 con 0x55a700421a10 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 10 client.26437 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 10 client.26437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:31.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 20 client.26437 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700430660 0x55a7004ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 4 client.26437 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-01T04:26:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.370+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a700393540 con 0x55a700430660 2024-08-01T04:26:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.371+0000 7f7eda000640 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700430660 0x55a7004ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:31.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.371+0000 7f7ebbe00640 10 client.26437 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.371+0000 7f7eda000640 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700430660 0x55a7004ad350 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dec358465aca2ec server_cookie=7469de19d343412c in_seq=0 out_seq=0 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ebbe00640 1 -- 172.21.15.3:0/1097941161 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a700393540 con 0x55a700430660 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ebbe00640 10 client.26437 handle_command_reply: tid=0 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 client.26437 shutdown 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 msgr2=0x55a700443570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 0x55a700443570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc09a100 tx=0x7f7ecc09b6e0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 shutdown_connections 2024-08-01T04:26:31.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55a700430660 0x55a7004ad350 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a700425da0 0x55a700443ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a700425480 0x55a700443ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 --2- 172.21.15.3:0/1097941161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a700421a10 0x55a700443570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 >> 172.21.15.3:0/1097941161 conn(0x55a70042de80 msgr2=0x55a70042c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ebbe00640 0 client.26437 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:31.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 shutdown_connections 2024-08-01T04:26:31.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 1 -- 172.21.15.3:0/1097941161 wait complete. 2024-08-01T04:26:31.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.373+0000 7f7ee3fc9740 20 client.26437 trim_cache size 0 max 16384 2024-08-01T04:26:31.717 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:31.717 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-08-01T04:26:31.717 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "3de851d5-16ea-4bac-9eba-6668add0a697", 2024-08-01T04:26:31.718 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-08-01T04:26:31.718 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:31.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7ec00a2cf0 msgr2=0x7f7ec00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7ec00a2cf0 0x7f7ec00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8000c00 tx=0x7f7ec80040c0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 msgr2=0x7f7edc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:31.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 0x7f7edc19a440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00e9f0 tx=0x7f7ecc00edf0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 shutdown_connections 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7edc10e1b0 0x7f7edc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7edc100fb0 0x7f7edc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f7ec00a2cf0 0x7f7ec00a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 --2- 172.21.15.3:0/2777299849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc100600 0x7f7edc19a440 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 >> 172.21.15.3:0/2777299849 conn(0x7f7edc0fc4d0 msgr2=0x7f7edc0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 shutdown_connections 2024-08-01T04:26:31.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:31.374+0000 7f7ee1600640 1 -- 172.21.15.3:0/2777299849 wait complete. 2024-08-01T04:26:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:31 smithi163 ceph-mon[37383]: pgmap v318: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-08-01T04:26:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:31 smithi163 ceph-mon[37383]: scrub queued for path: / 2024-08-01T04:26:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:31 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:31 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:26:32.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:31 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:32.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:31 smithi003 ceph-mon[32262]: pgmap v318: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-08-01T04:26:32.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:31 smithi003 ceph-mon[32262]: scrub queued for path: / 2024-08-01T04:26:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:31 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:31 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:26:32.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:31 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:32.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:31 smithi160 ceph-mon[37224]: pgmap v318: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-08-01T04:26:32.238 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:31 smithi160 ceph-mon[37224]: scrub queued for path: / 2024-08-01T04:26:32.238 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:31 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:32.238 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:31 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:26:32.238 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:31 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:32.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-01T04:26:33.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:26:33.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:26:33.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:26:33.036 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: active paths [/] 2024-08-01T04:26:33.037 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:32 smithi003 ceph-mon[32262]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:26:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:26:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:26:33.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: active paths [/] 2024-08-01T04:26:33.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:32 smithi163 ceph-mon[37383]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: active paths [/] 2024-08-01T04:26:33.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:32 smithi160 ceph-mon[37224]: scrub summary: idle+waiting paths [/] 2024-08-01T04:26:34.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:33 smithi163 ceph-mon[37383]: pgmap v319: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:34.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:33 smithi003 ceph-mon[32262]: pgmap v319: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:34.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:33 smithi160 ceph-mon[37224]: pgmap v319: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:36.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:35 smithi163 ceph-mon[37383]: pgmap v320: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:26:36.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:35 smithi003 ceph-mon[32262]: pgmap v320: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:26:36.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:35 smithi160 ceph-mon[37224]: pgmap v320: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 -- 172.21.15.3:0/1482717945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 msgr2=0x7f6660101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 --2- 172.21.15.3:0/1482717945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f6660101c70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6648009930 tx=0x7f664802a440 comp rx=0 tx=0).stop 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 -- 172.21.15.3:0/1482717945 shutdown_connections 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 --2- 172.21.15.3:0/1482717945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66601021b0 0x7f666010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 --2- 172.21.15.3:0/1482717945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f6660101c70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 --2- 172.21.15.3:0/1482717945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6660107820 0x7f6660107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 -- 172.21.15.3:0/1482717945 >> 172.21.15.3:0/1482717945 conn(0x7f66600fd510 msgr2=0x7f66600ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:36.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.522+0000 7f6667800640 1 -- 172.21.15.3:0/1482717945 shutdown_connections 2024-08-01T04:26:36.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.523+0000 7f6667800640 1 -- 172.21.15.3:0/1482717945 wait complete. 2024-08-01T04:26:36.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.524+0000 7f6667800640 1 Processor -- start 2024-08-01T04:26:36.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.524+0000 7f6667800640 1 -- start start 2024-08-01T04:26:36.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6667800640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f666019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:36.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6667800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66601021b0 0x7f666019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:36.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6667800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6660107820 0x7f666019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:36.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6667800640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111cb0 con 0x7f6660101810 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6667800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111b30 con 0x7f66601021b0 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6667800640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6660111e30 con 0x7f6660107820 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6664e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f666019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6664e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f666019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41496/0 (socket says 172.21.15.3:41496) 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6664e00640 1 -- 172.21.15.3:0/2243845349 learned_addr learned my addr 172.21.15.3:0/2243845349 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6665800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6660107820 0x7f666019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.525+0000 7f6657e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66601021b0 0x7f666019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:36.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6664e00640 1 -- 172.21.15.3:0/2243845349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6660107820 msgr2=0x7f666019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:36.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6664e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6660107820 0x7f666019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6664e00640 1 -- 172.21.15.3:0/2243845349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66601021b0 msgr2=0x7f666019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6664e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66601021b0 0x7f666019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6664e00640 1 -- 172.21.15.3:0/2243845349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666019f350 con 0x7f6660101810 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6657e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66601021b0 0x7f666019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6665800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6660107820 0x7f666019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6664e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f666019a360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f665000d950 tx=0x7f665000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:36.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650017070 con 0x7f6660101810 2024-08-01T04:26:36.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650009c30 con 0x7f6660101810 2024-08-01T04:26:36.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000f6b0 con 0x7f6660101810 2024-08-01T04:26:36.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.526+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666019f640 con 0x7f6660101810 2024-08-01T04:26:36.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.527+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601a6ed0 con 0x7f6660101810 2024-08-01T04:26:36.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.528+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6660101c70 con 0x7f6660101810 2024-08-01T04:26:36.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.530+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f66500109d0 con 0x7f6660101810 2024-08-01T04:26:36.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.532+0000 7f665e000640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f66440a2da0 0x7f66440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:36.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.532+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f66500cf0c0 con 0x7f6660101810 2024-08-01T04:26:36.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.532+0000 7f6657e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f66440a2da0 0x7f66440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:36.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.534+0000 7f6657e00640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f66440a2da0 0x7f66440a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6648006010 tx=0x7f6648005a20 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:36.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.537+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6650009de0 con 0x7f6660101810 2024-08-01T04:26:36.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.772+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f666010d1e0 con 0x7f66440a2da0 2024-08-01T04:26:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.786+0000 7f665e000640 1 -- 172.21.15.3:0/2243845349 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f666010d1e0 con 0x7f66440a2da0 2024-08-01T04:26:36.790 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-01T00:00:00", "created": "2024-08-01T04:16:30", "first": "2024-08-01T04:17:00", "last": "2024-08-01T04:26:00", "last_pruned": "2024-08-01T04:26:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-08-01T04:26:36.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f66440a2da0 msgr2=0x7f66440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:36.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f66440a2da0 0x7f66440a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6648006010 tx=0x7f6648005a20 comp rx=0 tx=0).stop 2024-08-01T04:26:36.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 msgr2=0x7f666019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:36.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f666019a360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f665000d950 tx=0x7f665000de10 comp rx=0 tx=0).stop 2024-08-01T04:26:36.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 shutdown_connections 2024-08-01T04:26:36.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f66440a2da0 0x7f66440a5260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6660107820 0x7f666019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66601021b0 0x7f666019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 --2- 172.21.15.3:0/2243845349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6660101810 0x7f666019a360 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:36.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 >> 172.21.15.3:0/2243845349 conn(0x7f66600fd510 msgr2=0x7f666010cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:36.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 shutdown_connections 2024-08-01T04:26:36.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:36.789+0000 7f6667800640 1 -- 172.21.15.3:0/2243845349 wait complete. 2024-08-01T04:26:36.815 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-01T04:26:36.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-01T04:26:37.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 -- 172.21.15.3:0/3054541955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 msgr2=0x7fd820101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:37.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 --2- 172.21.15.3:0/3054541955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd820101a60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd808009890 tx=0x7fd80802a360 comp rx=0 tx=0).stop 2024-08-01T04:26:37.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 -- 172.21.15.3:0/3054541955 shutdown_connections 2024-08-01T04:26:37.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 --2- 172.21.15.3:0/3054541955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 0x7fd82010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 --2- 172.21.15.3:0/3054541955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd820101a60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 --2- 172.21.15.3:0/3054541955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd820107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 -- 172.21.15.3:0/3054541955 >> 172.21.15.3:0/3054541955 conn(0x7fd8200fd300 msgr2=0x7fd8200ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:37.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 -- 172.21.15.3:0/3054541955 shutdown_connections 2024-08-01T04:26:37.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.035+0000 7fd825c00640 1 -- 172.21.15.3:0/3054541955 wait complete. 2024-08-01T04:26:37.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.036+0000 7fd825c00640 1 Processor -- start 2024-08-01T04:26:37.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.036+0000 7fd825c00640 1 -- start start 2024-08-01T04:26:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd82019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 0x7fd82019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd82019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd820111a00 con 0x7fd820107680 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd820111880 con 0x7fd820101fa0 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fd820111b80 con 0x7fd8201015e0 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd82019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd82019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41506/0 (socket says 172.21.15.3:41506) 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825200640 1 -- 172.21.15.3:0/1619982519 learned_addr learned my addr 172.21.15.3:0/1619982519 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd81fe00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd82019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd81f400640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 0x7fd82019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825200640 1 -- 172.21.15.3:0/1619982519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 msgr2=0x7fd82019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.037+0000 7fd825200640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd82019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd825200640 1 -- 172.21.15.3:0/1619982519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 msgr2=0x7fd82019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd825200640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 0x7fd82019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd825200640 1 -- 172.21.15.3:0/1619982519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82019f0f0 con 0x7fd820107680 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd81fe00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd82019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd81f400640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 0x7fd82019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd825200640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd82019e9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd81800eb10 tx=0x7fd81800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818004b20 con 0x7fd820107680 2024-08-01T04:26:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82019f3e0 con 0x7fd820107680 2024-08-01T04:26:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd818004cc0 con 0x7fd820107680 2024-08-01T04:26:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818010760 con 0x7fd820107680 2024-08-01T04:26:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.038+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8201a6c70 con 0x7fd820107680 2024-08-01T04:26:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.040+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd820101a60 con 0x7fd820107680 2024-08-01T04:26:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.042+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd818005020 con 0x7fd820107680 2024-08-01T04:26:37.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:36 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:26:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.043+0000 7fd81cc00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd8040a2d70 0x7fd8040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.044+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7fd8180cfb40 con 0x7fd820107680 2024-08-01T04:26:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.044+0000 7fd81fe00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd8040a2d70 0x7fd8040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.044+0000 7fd81fe00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd8040a2d70 0x7fd8040a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd8100096c0 tx=0x7fd810009260 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:37.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:36 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:26:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.047+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd81809c690 con 0x7fd820107680 2024-08-01T04:26:37.053 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:36 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:26:37.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.283+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 --> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd8200feff0 con 0x7fd8040a2d70 2024-08-01T04:26:37.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.297+0000 7fd81cc00640 1 -- 172.21.15.3:0/1619982519 <== mgr.14928 v2:172.21.15.3:6800/1464189073 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd8200feff0 con 0x7fd8040a2d70 2024-08-01T04:26:37.299 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-08-01T04:26:37.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.299+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd8040a2d70 msgr2=0x7fd8040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:37.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.299+0000 7fd825c00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd8040a2d70 0x7fd8040a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd8100096c0 tx=0x7fd810009260 comp rx=0 tx=0).stop 2024-08-01T04:26:37.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.299+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 msgr2=0x7fd82019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:37.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.299+0000 7fd825c00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd82019e9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd81800eb10 tx=0x7fd81800c2d0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 shutdown_connections 2024-08-01T04:26:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd820107680 0x7fd82019e9c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd820101fa0 0x7fd82019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fd8040a2d70 0x7fd8040a5230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 --2- 172.21.15.3:0/1619982519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8201015e0 0x7fd82019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 >> 172.21.15.3:0/1619982519 conn(0x7fd8200fd300 msgr2=0x7fd8200fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 shutdown_connections 2024-08-01T04:26:37.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:37.300+0000 7fd825c00640 1 -- 172.21.15.3:0/1619982519 wait complete. 2024-08-01T04:26:37.329 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-01T04:26:37.339 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-01T04:26:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:38.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:37 smithi003 ceph-mon[32262]: pgmap v321: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:26:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:37 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:26:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:37 smithi003 ceph-mon[32262]: scrub summary: idle 2024-08-01T04:26:38.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:37 smithi003 ceph-mon[32262]: from='client.16284 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:26:38.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:37 smithi160 ceph-mon[37224]: pgmap v321: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:26:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:37 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:26:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:37 smithi160 ceph-mon[37224]: scrub summary: idle 2024-08-01T04:26:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:37 smithi160 ceph-mon[37224]: from='client.16284 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:26:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:37 smithi163 ceph-mon[37383]: pgmap v321: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-08-01T04:26:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:37 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:26:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:37 smithi163 ceph-mon[37383]: scrub summary: idle 2024-08-01T04:26:38.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:37 smithi163 ceph-mon[37383]: from='client.16284 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:26:39.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 -- 172.21.15.3:0/3521014942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370100670 msgr2=0x7f3370100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 --2- 172.21.15.3:0/3521014942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370100670 0x7f3370100a50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3360009900 tx=0x7f336002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 -- 172.21.15.3:0/3521014942 shutdown_connections 2024-08-01T04:26:39.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 --2- 172.21.15.3:0/3521014942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f337010e220 0x7f3370110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 --2- 172.21.15.3:0/3521014942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3370101020 0x7f337010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 --2- 172.21.15.3:0/3521014942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370100670 0x7f3370100a50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 -- 172.21.15.3:0/3521014942 >> 172.21.15.3:0/3521014942 conn(0x7f33700fc500 msgr2=0x7f33700fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 -- 172.21.15.3:0/3521014942 shutdown_connections 2024-08-01T04:26:39.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.007+0000 7f3376200640 1 -- 172.21.15.3:0/3521014942 wait complete. 2024-08-01T04:26:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.008+0000 7f3376200640 1 Processor -- start 2024-08-01T04:26:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.008+0000 7f3376200640 1 -- start start 2024-08-01T04:26:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f3376200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3370100670 0x7f337019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f3376200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 0x7f337019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f3376200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f337010e220 0x7f337019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f3376200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3370113f20 con 0x7f3370101020 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f3376200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3370113da0 con 0x7f337010e220 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f3376200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f33701140a0 con 0x7f3370100670 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f336ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 0x7f337019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f336ea00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 0x7f337019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41532/0 (socket says 172.21.15.3:41532) 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f336f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3370100670 0x7f337019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f336ea00640 1 -- 172.21.15.3:0/2988819830 learned_addr learned my addr 172.21.15.3:0/2988819830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:39.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.009+0000 7f336fe00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f337010e220 0x7f337019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336ea00640 1 -- 172.21.15.3:0/2988819830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3370100670 msgr2=0x7f337019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336ea00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3370100670 0x7f337019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336ea00640 1 -- 172.21.15.3:0/2988819830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f337010e220 msgr2=0x7f337019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336ea00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f337010e220 0x7f337019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336ea00640 1 -- 172.21.15.3:0/2988819830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337019f350 con 0x7f3370101020 2024-08-01T04:26:39.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336ea00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 0x7f337019a8e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9f0 tx=0x7f335c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.010+0000 7f336f400640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3370100670 0x7f337019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.011+0000 7f3367e00640 1 -- 172.21.15.3:0/2988819830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c00cd80 con 0x7f3370101020 2024-08-01T04:26:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.011+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337019f640 con 0x7f3370101020 2024-08-01T04:26:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.011+0000 7f3367e00640 1 -- 172.21.15.3:0/2988819830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c010910 con 0x7f3370101020 2024-08-01T04:26:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.011+0000 7f3367e00640 1 -- 172.21.15.3:0/2988819830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c018950 con 0x7f3370101020 2024-08-01T04:26:39.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.011+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33701a6f20 con 0x7f3370101020 2024-08-01T04:26:39.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 Processor -- start 2024-08-01T04:26:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 -- start start 2024-08-01T04:26:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b6b90 0x559d7c0b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 0x559d7c0c1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0c1d70 0x559d7c0c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559d7c01cbf0 con 0x559d7c0b74b0 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x559d7bfb4bd0 con 0x559d7c0b6b90 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.015+0000 7f3378e75740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559d7c018bb0 con 0x559d7c0c1d70 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 0x559d7c0c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 0x559d7c0c1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41542/0 (socket says 172.21.15.3:41542) 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 -- 172.21.15.3:0/4146851523 learned_addr learned my addr 172.21.15.3:0/4146851523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:39.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336fe00640 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b6b90 0x559d7c0b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 -- 172.21.15.3:0/4146851523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0c1d70 msgr2=0x559d7c0c4160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0c1d70 0x559d7c0c4160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 -- 172.21.15.3:0/4146851523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b6b90 msgr2=0x559d7c0b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b6b90 0x559d7c0b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.016+0000 7f336f400640 1 -- 172.21.15.3:0/4146851523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7bfbb8a0 con 0x559d7c0b74b0 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.017+0000 7f336f400640 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 0x559d7c0c1830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3360009900 tx=0x7f3360005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362d6578640cb934 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.017+0000 7f335be00640 1 -- 172.21.15.3:0/4146851523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360035040 con 0x559d7c0b74b0 2024-08-01T04:26:39.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.017+0000 7f335be00640 1 -- 172.21.15.3:0/4146851523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3360004760 con 0x559d7c0b74b0 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.017+0000 7f335be00640 1 -- 172.21.15.3:0/4146851523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360003970 con 0x559d7c0b74b0 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 -- 172.21.15.3:0/4146851523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 msgr2=0x559d7c0c1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 0x559d7c0c1830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3360009900 tx=0x7f3360005de0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 -- 172.21.15.3:0/4146851523 shutdown_connections 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0c1d70 0x559d7c0c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b74b0 0x559d7c0c1830 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 --2- 172.21.15.3:0/4146851523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b6b90 0x559d7c0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 -- 172.21.15.3:0/4146851523 >> 172.21.15.3:0/4146851523 conn(0x559d7c0b3120 msgr2=0x559d7c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:39.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3367e00640 1 -- 172.21.15.3:0/2988819830 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f335c010ac0 con 0x7f3370101020 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 -- 172.21.15.3:0/4146851523 shutdown_connections 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.018+0000 7f3378e75740 1 -- 172.21.15.3:0/4146851523 wait complete. 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.019+0000 7f3378e75740 1 -- start start 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.019+0000 7f3378e75740 1 Processor -- start 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0bedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559d7c0c4940 con 0x559d7c0b6b90 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x559d7bfd7d40 con 0x559d7c0bf870 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x559d7c0c4ac0 con 0x559d7c0b74b0 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.020+0000 7f3378e75740 10 client.0 _ll_register_callbacks cb 0x559d7bfa0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0bedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f3367e00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f33540a2c00 0x7f33540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54244/0 (socket says 172.21.15.3:54244) 2024-08-01T04:26:39.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 -- 172.21.15.3:0/1856635495 learned_addr learned my addr 172.21.15.3:0/1856635495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:39.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336ea00640 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f3367e00640 1 -- 172.21.15.3:0/2988819830 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f335c0cf950 con 0x7f3370101020 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336fe00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f33540a2c00 0x7f33540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 -- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 msgr2=0x559d7c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 -- 172.21.15.3:0/1856635495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 msgr2=0x559d7c0bedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0bedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.021+0000 7f336f400640 1 -- 172.21.15.3:0/1856635495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7bd8e3c0 con 0x559d7c0b74b0 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.022+0000 7f336f400640 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bf330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3360004b40 tx=0x7f3360000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30370bdd9afc7b48 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:39.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.022+0000 7f3358c00640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.022+0000 7f336fe00640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f33540a2c00 0x7f33540a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3368005c40 tx=0x7f3368015040 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.022+0000 7f3358c00640 1 -- 172.21.15.3:0/1856635495 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360053060 con 0x559d7c0b74b0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.022+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360004760 con 0x559d7c0b74b0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.022+0000 7f3358c00640 1 -- 172.21.15.3:0/1856635495 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f3360005210 con 0x559d7c0b74b0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3358c00640 1 -- 172.21.15.3:0/1856635495 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3360035050 con 0x559d7c0b74b0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 10 client.26458 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d7c0c4940 con 0x559d7c0b74b0 2024-08-01T04:26:39.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3358c00640 1 -- 172.21.15.3:0/1856635495 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3360004760 con 0x559d7c0b74b0 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3358c00640 1 -- 172.21.15.3:0/1856635495 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f33600315b0 con 0x559d7c0b74b0 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 10 client.26458 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 20 client.26458 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c0d1cd0 0x559d7c0d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 4 client.26458 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.023+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d7c02a160 con 0x559d7c0d1cd0 2024-08-01T04:26:39.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.024+0000 7f336ea00640 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c0d1cd0 0x559d7c0d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.024+0000 7f336ea00640 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c0d1cd0 0x559d7c0d40c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda877e0880d5820 server_cookie=5d4e5f4a936c52fd in_seq=0 out_seq=0 2024-08-01T04:26:39.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.024+0000 7f3358c00640 10 client.26458 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:39.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.025+0000 7f3358c00640 1 -- 172.21.15.3:0/1856635495 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d7c02a160 con 0x559d7c0d1cd0 2024-08-01T04:26:39.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.025+0000 7f3358c00640 10 client.26458 handle_command_reply: tid=0 2024-08-01T04:26:39.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.025+0000 7f3378e75740 1 client.26458 shutdown 2024-08-01T04:26:39.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.025+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 msgr2=0x559d7c0bf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.025+0000 7f3378e75740 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bf330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3360004b40 tx=0x7f3360000bc0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 shutdown_connections 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c0d1cd0 0x559d7c0d40c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bf330 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 --2- 172.21.15.3:0/1856635495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0bedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3358c00640 0 client.26458 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 >> 172.21.15.3:0/1856635495 conn(0x559d7c0b3120 msgr2=0x559d7c0b8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:39.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 shutdown_connections 2024-08-01T04:26:39.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 1 -- 172.21.15.3:0/1856635495 wait complete. 2024-08-01T04:26:39.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.026+0000 7f3378e75740 20 client.26458 trim_cache size 0 max 16384 2024-08-01T04:26:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.045+0000 7f3378e75740 1 Processor -- start 2024-08-01T04:26:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.045+0000 7f3378e75740 1 -- start start 2024-08-01T04:26:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f3378e75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f3378e75740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 0x559d7c0d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f3378e75740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0bf870 0x559d7c0d9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f3378e75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33701b2350 con 0x559d7c0b6b90 2024-08-01T04:26:39.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f3378e75740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f33701afc20 con 0x559d7c0b74b0 2024-08-01T04:26:39.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f3378e75740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f33701ada90 con 0x559d7c0bf870 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f336ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 0x559d7c0d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f336fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0bf870 0x559d7c0d9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f336ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 0x559d7c0d96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:50026/0 (socket says 172.21.15.3:50026) 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f336ea00640 1 -- 172.21.15.3:0/3527466704 learned_addr learned my addr 172.21.15.3:0/3527466704 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.046+0000 7f336f400640 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336ea00640 1 -- 172.21.15.3:0/3527466704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0bf870 msgr2=0x559d7c0d9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336ea00640 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0bf870 0x559d7c0d9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336ea00640 1 -- 172.21.15.3:0/3527466704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 msgr2=0x559d7c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336ea00640 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336ea00640 1 -- 172.21.15.3:0/3527466704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7c0bfd90 con 0x559d7c0b74b0 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336f400640 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.047+0000 7f336ea00640 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 0x559d7c0d96c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9c0 tx=0x7f335c06de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189841ce85b00c12 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.048+0000 7f335b400640 1 -- 172.21.15.3:0/3527466704 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0a4a70 con 0x559d7c0b74b0 2024-08-01T04:26:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.048+0000 7f335b400640 1 -- 172.21.15.3:0/3527466704 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c099ea0 con 0x559d7c0b74b0 2024-08-01T04:26:39.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.048+0000 7f335b400640 1 -- 172.21.15.3:0/3527466704 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0a45e0 con 0x559d7c0b74b0 2024-08-01T04:26:39.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 -- 172.21.15.3:0/3527466704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 msgr2=0x559d7c0d96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 0x559d7c0d96c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9c0 tx=0x7f335c06de80 comp rx=0 tx=0).stop 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 -- 172.21.15.3:0/3527466704 shutdown_connections 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0bf870 0x559d7c0d9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0b74b0 0x559d7c0d96c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 --2- 172.21.15.3:0/3527466704 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 -- 172.21.15.3:0/3527466704 >> 172.21.15.3:0/3527466704 conn(0x559d7c0b3120 msgr2=0x559d7c0bdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 -- 172.21.15.3:0/3527466704 shutdown_connections 2024-08-01T04:26:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.049+0000 7f3378e75740 1 -- 172.21.15.3:0/3527466704 wait complete. 2024-08-01T04:26:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.050+0000 7f3378e75740 1 -- start start 2024-08-01T04:26:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.050+0000 7f3378e75740 1 Processor -- start 2024-08-01T04:26:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33701ae0e0 con 0x559d7c0b6b90 2024-08-01T04:26:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f33701af550 con 0x559d7c0bf870 2024-08-01T04:26:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f33701ae7b0 con 0x559d7c0b74b0 2024-08-01T04:26:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f3378e75740 10 client.0 _ll_register_callbacks cb 0x7f3368009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f336ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f336f400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f336ea00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:54266/0 (socket says 172.21.15.3:54266) 2024-08-01T04:26:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f336ea00640 1 -- 172.21.15.3:0/2306595170 learned_addr learned my addr 172.21.15.3:0/2306595170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.051+0000 7f336fe00640 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f336f400640 1 -- 172.21.15.3:0/2306595170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 msgr2=0x559d7c0bbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f336f400640 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f336f400640 1 -- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 msgr2=0x559d7c0bc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f336f400640 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0bc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f336f400640 1 -- 172.21.15.3:0/2306595170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7c0bc8d0 con 0x559d7c0b6b90 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f336f400640 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0ca460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3360004fd0 tx=0x7f3360005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cc5b752be2a826 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f3358c00640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.052+0000 7f3358c00640 1 -- 172.21.15.3:0/2306595170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360053040 con 0x559d7c0b6b90 2024-08-01T04:26:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3358c00640 1 -- 172.21.15.3:0/2306595170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f3360005210 con 0x559d7c0b6b90 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f335c099ea0 con 0x559d7c0b6b90 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3358c00640 1 -- 172.21.15.3:0/2306595170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3360004760 con 0x559d7c0b6b90 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3378e75740 10 client.16308 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d7c13c730 con 0x559d7c0b6b90 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3358c00640 1 -- 172.21.15.3:0/2306595170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c099ea0 con 0x559d7c0b6b90 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.053+0000 7f3358c00640 1 -- 172.21.15.3:0/2306595170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f3360003ad0 con 0x559d7c0b6b90 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f3378e75740 10 client.16308 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f3378e75740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f3378e75740 20 client.16308 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f3378e75740 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c13cc90 0x559d7c13f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f3378e75740 4 client.16308 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d7c024670 con 0x559d7c13cc90 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.054+0000 7f336ea00640 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c13cc90 0x559d7c13f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f336ea00640 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c13cc90 0x559d7c13f0b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3ee0e70344fd69 server_cookie=86022fc6db16b7e1 in_seq=0 out_seq=0 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3358c00640 10 client.16308 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3358c00640 1 -- 172.21.15.3:0/2306595170 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d7c024670 con 0x559d7c13cc90 2024-08-01T04:26:39.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3358c00640 10 client.16308 handle_command_reply: tid=0 2024-08-01T04:26:39.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 client.16308 shutdown 2024-08-01T04:26:39.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:38 smithi003 ceph-mon[32262]: from='client.16290 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:26:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 msgr2=0x559d7c0ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0ca460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3360004fd0 tx=0x7f3360005de0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 shutdown_connections 2024-08-01T04:26:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x559d7c13cc90 0x559d7c13f0b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559d7c0bf870 0x559d7c0bc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559d7c0b74b0 0x559d7c0bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 --2- 172.21.15.3:0/2306595170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d7c0b6b90 0x559d7c0ca460 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.055+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 >> 172.21.15.3:0/2306595170 conn(0x559d7c0b3120 msgr2=0x559d7c0b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.056+0000 7f3358c00640 0 client.16308 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.056+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 shutdown_connections 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.056+0000 7f3378e75740 1 -- 172.21.15.3:0/2306595170 wait complete. 2024-08-01T04:26:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.056+0000 7f3378e75740 20 client.16308 trim_cache size 0 max 16384 2024-08-01T04:26:39.064 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:39.064 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-08-01T04:26:39.064 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-08-01T04:26:39.064 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f33540a2c00 msgr2=0x7f33540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f33540a2c00 0x7f33540a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3368005c40 tx=0x7f3368015040 comp rx=0 tx=0).stop 2024-08-01T04:26:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 msgr2=0x7f337019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 0x7f337019a8e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f335c00e9f0 tx=0x7f335c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 shutdown_connections 2024-08-01T04:26:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f337010e220 0x7f337019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3370101020 0x7f337019a8e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f33540a2c00 0x7f33540a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 --2- 172.21.15.3:0/2988819830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3370100670 0x7f337019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 >> 172.21.15.3:0/2988819830 conn(0x7f33700fc500 msgr2=0x7f33700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 shutdown_connections 2024-08-01T04:26:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:39.057+0000 7f3376200640 1 -- 172.21.15.3:0/2988819830 wait complete. 2024-08-01T04:26:39.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:38 smithi160 ceph-mon[37224]: from='client.16290 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:26:39.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:38 smithi163 ceph-mon[37383]: from='client.16290 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-01T04:26:39.882 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-01T04:26:39.882 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:26:40.148 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:39 smithi003 ceph-mon[32262]: pgmap v322: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:26:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:39 smithi160 ceph-mon[37224]: pgmap v322: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:26:40.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:39 smithi163 ceph-mon[37383]: pgmap v322: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-08-01T04:26:40.348 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:41.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 -- 172.21.15.3:0/1100461045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5970103900 msgr2=0x7f5970103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:41.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 --2- 172.21.15.3:0/1100461045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5970103900 0x7f5970103d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f595c009930 tx=0x7f595c02a420 comp rx=0 tx=0).stop 2024-08-01T04:26:41.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 -- 172.21.15.3:0/1100461045 shutdown_connections 2024-08-01T04:26:41.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 --2- 172.21.15.3:0/1100461045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59701042a0 0x7f59701107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 --2- 172.21.15.3:0/1100461045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5970103900 0x7f5970103d60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 --2- 172.21.15.3:0/1100461045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5970109900 0x7f5970109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 -- 172.21.15.3:0/1100461045 >> 172.21.15.3:0/1100461045 conn(0x7f59700ff600 msgr2=0x7f5970101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:41.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 -- 172.21.15.3:0/1100461045 shutdown_connections 2024-08-01T04:26:41.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.026+0000 7f596fe00640 1 -- 172.21.15.3:0/1100461045 wait complete. 2024-08-01T04:26:41.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 Processor -- start 2024-08-01T04:26:41.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 -- start start 2024-08-01T04:26:41.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970103900 0x7f597019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:41.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59701042a0 0x7f597019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:41.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5970109900 0x7f597019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:41.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5970113d00 con 0x7f59701042a0 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5970113b80 con 0x7f5970103900 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.028+0000 7f596fe00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f5970113e80 con 0x7f5970109900 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59701042a0 0x7f597019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970103900 0x7f597019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596e000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59701042a0 0x7f597019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41588/0 (socket says 172.21.15.3:41588) 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596e000640 1 -- 172.21.15.3:0/1929266562 learned_addr learned my addr 172.21.15.3:0/1929266562 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:41.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596f400640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5970109900 0x7f597019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596ea00640 1 -- 172.21.15.3:0/1929266562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5970109900 msgr2=0x7f597019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596ea00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5970109900 0x7f597019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596ea00640 1 -- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59701042a0 msgr2=0x7f597019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596ea00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59701042a0 0x7f597019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.029+0000 7f596ea00640 1 -- 172.21.15.3:0/1929266562 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597019f1d0 con 0x7f5970103900 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.030+0000 7f596ea00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970103900 0x7f597019a220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9f0 tx=0x7f596000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.030+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596000ccc0 con 0x7f5970103900 2024-08-01T04:26:41.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.030+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5960005020 con 0x7f5970103900 2024-08-01T04:26:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.030+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59600049e0 con 0x7f5970103900 2024-08-01T04:26:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.030+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597019f4c0 con 0x7f5970103900 2024-08-01T04:26:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.030+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59701a6da0 con 0x7f5970103900 2024-08-01T04:26:41.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.031+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5970103d60 con 0x7f5970103900 2024-08-01T04:26:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.033+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5960005560 con 0x7f5970103900 2024-08-01T04:26:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.035+0000 7f5967400640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f59540a2d70 0x7f59540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.035+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f59600cf340 con 0x7f5970103900 2024-08-01T04:26:41.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.035+0000 7f596e000640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f59540a2d70 0x7f59540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:41.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.036+0000 7f596e000640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f59540a2d70 0x7f59540a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5970101c80 tx=0x7f595c009560 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:41.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.039+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f59600051d0 con 0x7f5970103900 2024-08-01T04:26:41.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.269+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597002d5e0 con 0x7f5970103900 2024-08-01T04:26:41.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.270+0000 7f5967400640 1 -- 172.21.15.3:0/1929266562 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f596009be90 con 0x7f5970103900 2024-08-01T04:26:41.273 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:26:41.273 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:26:41.273 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:26:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.273+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f59540a2d70 msgr2=0x7f59540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.273+0000 7f596fe00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f59540a2d70 0x7f59540a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5970101c80 tx=0x7f595c009560 comp rx=0 tx=0).stop 2024-08-01T04:26:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.273+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970103900 msgr2=0x7f597019a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.273+0000 7f596fe00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970103900 0x7f597019a220 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9f0 tx=0x7f596000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 shutdown_connections 2024-08-01T04:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f59540a2d70 0x7f59540a5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5970109900 0x7f597019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59701042a0 0x7f597019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 --2- 172.21.15.3:0/1929266562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970103900 0x7f597019a220 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 >> 172.21.15.3:0/1929266562 conn(0x7f59700ff600 msgr2=0x7f59700ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:41.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 shutdown_connections 2024-08-01T04:26:41.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:41.274+0000 7f596fe00640 1 -- 172.21.15.3:0/1929266562 wait complete. 2024-08-01T04:26:42.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:41 smithi003 ceph-mon[32262]: pgmap v323: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:26:42.038 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:41 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1929266562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:26:42.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-01T04:26:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:41 smithi160 ceph-mon[37224]: pgmap v323: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:26:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:41 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1929266562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:26:42.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:41 smithi163 ceph-mon[37383]: pgmap v323: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-01T04:26:42.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:41 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1929266562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:26:42.632 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.357+0000 7f0b32c00640 1 -- 172.21.15.3:0/1998656299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c1016e0 msgr2=0x7f0b2c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.357+0000 7f0b32c00640 1 --2- 172.21.15.3:0/1998656299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c101b40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009930 tx=0x7f0b1802a420 comp rx=0 tx=0).stop 2024-08-01T04:26:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 -- 172.21.15.3:0/1998656299 shutdown_connections 2024-08-01T04:26:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 --2- 172.21.15.3:0/1998656299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c102080 0x7f0b2c10e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 --2- 172.21.15.3:0/1998656299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c101b40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 --2- 172.21.15.3:0/1998656299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b2c1076e0 0x7f0b2c107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 -- 172.21.15.3:0/1998656299 >> 172.21.15.3:0/1998656299 conn(0x7f0b2c0fd3e0 msgr2=0x7f0b2c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 -- 172.21.15.3:0/1998656299 shutdown_connections 2024-08-01T04:26:43.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.358+0000 7f0b32c00640 1 -- 172.21.15.3:0/1998656299 wait complete. 2024-08-01T04:26:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 Processor -- start 2024-08-01T04:26:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 -- start start 2024-08-01T04:26:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c102080 0x7f0b2c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b2c1076e0 0x7f0b2c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c111b30 con 0x7f0b2c102080 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.359+0000 7f0b32c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1119b0 con 0x7f0b2c1016e0 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b32c00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c111cb0 con 0x7f0b2c1076e0 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34254/0 (socket says 172.21.15.3:34254) 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 -- 172.21.15.3:0/2640107242 learned_addr learned my addr 172.21.15.3:0/2640107242 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b30e00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c102080 0x7f0b2c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b32200640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b2c1076e0 0x7f0b2c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 -- 172.21.15.3:0/2640107242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b2c1076e0 msgr2=0x7f0b2c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b2c1076e0 0x7f0b2c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 -- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c102080 msgr2=0x7f0b2c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c102080 0x7f0b2c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.360+0000 7f0b31800640 1 -- 172.21.15.3:0/2640107242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c19f1d0 con 0x7f0b2c1016e0 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.361+0000 7f0b31800640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c19a220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000e9f0 tx=0x7f0b2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.361+0000 7f0b2a000640 1 -- 172.21.15.3:0/2640107242 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000cd80 con 0x7f0b2c1016e0 2024-08-01T04:26:43.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.361+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c19f4c0 con 0x7f0b2c1016e0 2024-08-01T04:26:43.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.361+0000 7f0b2a000640 1 -- 172.21.15.3:0/2640107242 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b20010910 con 0x7f0b2c1016e0 2024-08-01T04:26:43.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.361+0000 7f0b2a000640 1 -- 172.21.15.3:0/2640107242 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b200188f0 con 0x7f0b2c1016e0 2024-08-01T04:26:43.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.361+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1a6da0 con 0x7f0b2c1016e0 2024-08-01T04:26:43.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 Processor -- start 2024-08-01T04:26:43.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 -- start start 2024-08-01T04:26:43.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b2a000640 1 -- 172.21.15.3:0/2640107242 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0b20010ac0 con 0x7f0b2c1016e0 2024-08-01T04:26:43.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 0x55a5d7db2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a5d7d17bf0 con 0x55a5d7dbd8c0 2024-08-01T04:26:43.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a5d7cafb20 con 0x55a5d7db26e0 2024-08-01T04:26:43.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.365+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a5d7d13bb0 con 0x55a5d7db3000 2024-08-01T04:26:43.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b31800640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 0x55a5d7db2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 0x55a5d7db2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34262/0 (socket says 172.21.15.3:34262) 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 -- 172.21.15.3:0/3625224395 learned_addr learned my addr 172.21.15.3:0/3625224395 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b30e00640 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 -- 172.21.15.3:0/3625224395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 msgr2=0x55a5d7dbd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 -- 172.21.15.3:0/3625224395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7dbd8c0 msgr2=0x55a5d7dbfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 -- 172.21.15.3:0/3625224395 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d7cb67f0 con 0x55a5d7db26e0 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b30e00640 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.366+0000 7f0b32200640 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 0x55a5d7db2ac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009fd0 tx=0x7f0b240118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64c06bb996fb379 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.367+0000 7f0b1ea00640 1 -- 172.21.15.3:0/3625224395 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24004d20 con 0x55a5d7db26e0 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.367+0000 7f0b2a000640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0b140a2cf0 0x7f0b140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.367+0000 7f0b1ea00640 1 -- 172.21.15.3:0/3625224395 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b24004050 con 0x55a5d7db26e0 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.367+0000 7f0b1ea00640 1 -- 172.21.15.3:0/3625224395 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24005280 con 0x55a5d7db26e0 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.367+0000 7f0b2a000640 1 -- 172.21.15.3:0/2640107242 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f0b200cf880 con 0x7f0b2c1016e0 2024-08-01T04:26:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.367+0000 7f0b31800640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0b140a2cf0 0x7f0b140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3625224395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 msgr2=0x55a5d7db2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 0x55a5d7db2ac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009fd0 tx=0x7f0b240118d0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3625224395 shutdown_connections 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3625224395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7db26e0 0x55a5d7db2ac0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3625224395 >> 172.21.15.3:0/3625224395 conn(0x55a5d7daec70 msgr2=0x55a5d7daf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b31800640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0b140a2cf0 0x7f0b140a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cd010 tx=0x7f0b200d5550 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3625224395 shutdown_connections 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.368+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3625224395 wait complete. 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 -- start start 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 Processor -- start 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a5d7dc0490 con 0x55a5d7db26e0 2024-08-01T04:26:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55a5d7cd2cf0 con 0x55a5d7dbd8c0 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55a5d7dc0610 con 0x55a5d7db3000 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.369+0000 7f0b36c8d740 10 client.0 _ll_register_callbacks cb 0x55a5d7cbee30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b32200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b32200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dba7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46914/0 (socket says 172.21.15.3:46914) 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b32200640 1 -- 172.21.15.3:0/416572889 learned_addr learned my addr 172.21.15.3:0/416572889 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b30e00640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b32200640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b30e00640 1 -- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 msgr2=0x55a5d7dbb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b30e00640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b30e00640 1 -- 172.21.15.3:0/416572889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 msgr2=0x55a5d7dba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b30e00640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b30e00640 1 -- 172.21.15.3:0/416572889 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d7a893c0 con 0x55a5d7db3000 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.370+0000 7f0b32200640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.371+0000 7f0b30e00640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbad20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009230 tx=0x7f0b180052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477ea821faa74fc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:43.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.371+0000 7f0b13400640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.371+0000 7f0b13400640 1 -- 172.21.15.3:0/416572889 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18035030 con 0x55a5d7db3000 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.371+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b24004050 con 0x55a5d7db3000 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.371+0000 7f0b13400640 1 -- 172.21.15.3:0/416572889 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f0b180048c0 con 0x55a5d7db3000 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.371+0000 7f0b13400640 1 -- 172.21.15.3:0/416572889 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b18005d50 con 0x55a5d7db3000 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 10 client.26482 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5d7dcc1c0 con 0x55a5d7db3000 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b13400640 1 -- 172.21.15.3:0/416572889 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b24004050 con 0x55a5d7db3000 2024-08-01T04:26:43.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b13400640 1 -- 172.21.15.3:0/416572889 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0b18031b60 con 0x55a5d7db3000 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 10 client.26482 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 10 client.26482 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 20 client.26482 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7dcd5d0 0x55a5d7dcf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 4 client.26482 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.372+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d7d25160 con 0x55a5d7dcd5d0 2024-08-01T04:26:43.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.373+0000 7f0b32200640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7dcd5d0 0x55a5d7dcf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.373+0000 7f0b32200640 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7dcd5d0 0x55a5d7dcf9c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d384484055394d99 server_cookie=574e21e037ed0c8c in_seq=0 out_seq=0 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.373+0000 7f0b13400640 10 client.26482 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.374+0000 7f0b13400640 1 -- 172.21.15.3:0/416572889 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a5d7d25160 con 0x55a5d7dcd5d0 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.374+0000 7f0b13400640 10 client.26482 handle_command_reply: tid=0 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.374+0000 7f0b36c8d740 1 client.26482 shutdown 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 msgr2=0x55a5d7dbad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbad20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009230 tx=0x7f0b180052f0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 shutdown_connections 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dbb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7dcd5d0 0x55a5d7dcf9c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dbad20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/416572889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 >> 172.21.15.3:0/416572889 conn(0x55a5d7daec70 msgr2=0x55a5d7db4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b13400640 0 client.26482 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 shutdown_connections 2024-08-01T04:26:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 1 -- 172.21.15.3:0/416572889 wait complete. 2024-08-01T04:26:43.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.375+0000 7f0b36c8d740 20 client.26482 trim_cache size 0 max 16384 2024-08-01T04:26:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 Processor -- start 2024-08-01T04:26:43.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 -- start start 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7db5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dd5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dd5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1b2130 con 0x55a5d7db26e0 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1afa00 con 0x55a5d7dbd8c0 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.390+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1ad870 con 0x55a5d7db3000 2024-08-01T04:26:43.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b32200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7db5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b32200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7db5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46918/0 (socket says 172.21.15.3:46918) 2024-08-01T04:26:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b32200640 1 -- 172.21.15.3:0/3744663550 learned_addr learned my addr 172.21.15.3:0/3744663550 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b30e00640 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dd5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b32200640 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dd5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b30e00640 1 -- 172.21.15.3:0/3744663550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 msgr2=0x55a5d7dd5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b30e00640 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dd5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b30e00640 1 -- 172.21.15.3:0/3744663550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 msgr2=0x55a5d7db5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b30e00640 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7db5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.391+0000 7f0b30e00640 1 -- 172.21.15.3:0/3744663550 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d7dcfd30 con 0x55a5d7db3000 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.392+0000 7f0b32200640 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7db5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.392+0000 7f0b30e00640 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dd5250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b1802aa10 tx=0x7f0b18009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b19c7fceeca6a236 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.392+0000 7f0b1e000640 1 -- 172.21.15.3:0/3744663550 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18035070 con 0x55a5d7db3000 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.392+0000 7f0b1e000640 1 -- 172.21.15.3:0/3744663550 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1803f070 con 0x55a5d7db3000 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.392+0000 7f0b1e000640 1 -- 172.21.15.3:0/3744663550 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b180046c0 con 0x55a5d7db3000 2024-08-01T04:26:43.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3744663550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 msgr2=0x55a5d7dd5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dd5250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0b1802aa10 tx=0x7f0b18009830 comp rx=0 tx=0).stop 2024-08-01T04:26:43.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3744663550 shutdown_connections 2024-08-01T04:26:43.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7dd5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dd5250 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/3744663550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7db5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3744663550 >> 172.21.15.3:0/3744663550 conn(0x55a5d7daec70 msgr2=0x55a5d7daf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3744663550 shutdown_connections 2024-08-01T04:26:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 -- 172.21.15.3:0/3744663550 wait complete. 2024-08-01T04:26:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.393+0000 7f0b36c8d740 1 -- start start 2024-08-01T04:26:43.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 Processor -- start 2024-08-01T04:26:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dc5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dc6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7db7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1adec0 con 0x55a5d7db26e0 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1af330 con 0x55a5d7dbd8c0 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c1ae590 con 0x55a5d7db3000 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b36c8d740 10 client.0 _ll_register_callbacks cb 0x7f0b18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.394+0000 7f0b32200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dc5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dc5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46922/0 (socket says 172.21.15.3:46922) 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 -- 172.21.15.3:0/354129382 learned_addr learned my addr 172.21.15.3:0/354129382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:43.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b30e00640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dc6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7db7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 -- 172.21.15.3:0/354129382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 msgr2=0x55a5d7dc6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dc6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 -- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 msgr2=0x55a5d7db7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7db7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 -- 172.21.15.3:0/354129382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d7db7a50 con 0x55a5d7db26e0 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b32200640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dc5b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400a010 tx=0x7f0b24000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3d470c2315c710 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.395+0000 7f0b13400640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b13400640 1 -- 172.21.15.3:0/354129382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240044e0 con 0x55a5d7db26e0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b13400640 1 -- 172.21.15.3:0/354129382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f0b24004680 con 0x55a5d7db26e0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b1803f070 con 0x55a5d7db26e0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b13400640 1 -- 172.21.15.3:0/354129382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b24004880 con 0x55a5d7db26e0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b36c8d740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5d7e37e40 con 0x55a5d7db26e0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b13400640 1 -- 172.21.15.3:0/354129382 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1803f070 con 0x55a5d7db26e0 2024-08-01T04:26:43.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.396+0000 7f0b13400640 1 -- 172.21.15.3:0/354129382 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0b2403a020 con 0x55a5d7db26e0 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b36c8d740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b36c8d740 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b36c8d740 20 client.16332 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7e38420 0x55a5d7e3a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b36c8d740 4 client.16332 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 --> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5d7d1f670 con 0x55a5d7e38420 2024-08-01T04:26:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.397+0000 7f0b30e00640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7e38420 0x55a5d7e3a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b30e00640 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7e38420 0x55a5d7e3a840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0f349b2450cb9dc server_cookie=eae047fb29ae029d in_seq=0 out_seq=0 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b13400640 10 client.16332 ms_handle_connect on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b13400640 1 -- 172.21.15.3:0/354129382 <== mds.2 v2:172.21.15.160:6836/3127569351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5d7d1f670 con 0x55a5d7e38420 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b13400640 10 client.16332 handle_command_reply: tid=0 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 client.16332 shutdown 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 msgr2=0x55a5d7dc5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dc5b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400a010 tx=0x7f0b24000f40 comp rx=0 tx=0).stop 2024-08-01T04:26:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 shutdown_connections 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a5d7dbd8c0 0x55a5d7db7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.160:6836/3127569351,v1:172.21.15.160:6837/3127569351] conn(0x55a5d7e38420 0x55a5d7e3a840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a5d7db3000 0x55a5d7dc6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 --2- 172.21.15.3:0/354129382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a5d7db26e0 0x55a5d7dc5b40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 >> 172.21.15.3:0/354129382 conn(0x55a5d7daec70 msgr2=0x55a5d7dad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.398+0000 7f0b13400640 0 client.16332 ms_handle_reset on v2:172.21.15.160:6836/3127569351 2024-08-01T04:26:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 shutdown_connections 2024-08-01T04:26:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b36c8d740 1 -- 172.21.15.3:0/354129382 wait complete. 2024-08-01T04:26:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b36c8d740 20 client.16332 trim_cache size 0 max 16384 2024-08-01T04:26:43.408 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:26:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0b140a2cf0 msgr2=0x7f0b140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b32c00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0b140a2cf0 0x7f0b140a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0b200cd010 tx=0x7f0b200d5550 comp rx=0 tx=0).stop 2024-08-01T04:26:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 msgr2=0x7f0b2c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.399+0000 7f0b32c00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c19a220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000e9f0 tx=0x7f0b2000eeb0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 shutdown_connections 2024-08-01T04:26:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f0b140a2cf0 0x7f0b140a51b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b2c1076e0 0x7f0b2c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b2c102080 0x7f0b2c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 --2- 172.21.15.3:0/2640107242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b2c1016e0 0x7f0b2c19a220 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 >> 172.21.15.3:0/2640107242 conn(0x7f0b2c0fd3e0 msgr2=0x7f0b2c10c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 shutdown_connections 2024-08-01T04:26:43.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:43.400+0000 7f0b32c00640 1 -- 172.21.15.3:0/2640107242 wait complete. 2024-08-01T04:26:44.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:43 smithi160 ceph-mon[37224]: pgmap v324: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:44.230 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:43 smithi003 ceph-mon[32262]: pgmap v324: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:44.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:43 smithi163 ceph-mon[37383]: pgmap v324: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:44.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-01T04:26:44.808 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:45.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.460+0000 7f21ddc00640 1 -- 172.21.15.3:0/768988161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8103ac0 msgr2=0x7f21d8103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.460+0000 7f21ddc00640 1 --2- 172.21.15.3:0/768988161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8103ac0 0x7f21d8103f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009930 tx=0x7f21c402a420 comp rx=0 tx=0).stop 2024-08-01T04:26:45.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 -- 172.21.15.3:0/768988161 shutdown_connections 2024-08-01T04:26:45.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 --2- 172.21.15.3:0/768988161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 0x7f21d8110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 --2- 172.21.15.3:0/768988161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8103ac0 0x7f21d8103f20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 --2- 172.21.15.3:0/768988161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8109ac0 0x7f21d8109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 -- 172.21.15.3:0/768988161 >> 172.21.15.3:0/768988161 conn(0x7f21d80ff7c0 msgr2=0x7f21d8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:45.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 -- 172.21.15.3:0/768988161 shutdown_connections 2024-08-01T04:26:45.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.461+0000 7f21ddc00640 1 -- 172.21.15.3:0/768988161 wait complete. 2024-08-01T04:26:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.462+0000 7f21ddc00640 1 Processor -- start 2024-08-01T04:26:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.462+0000 7f21ddc00640 1 -- start start 2024-08-01T04:26:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21ddc00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8103ac0 0x7f21d819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21ddc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 0x7f21d819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21ddc00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8109ac0 0x7f21d819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21ddc00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8113ec0 con 0x7f21d8109ac0 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21ddc00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8113d40 con 0x7f21d8104460 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21ddc00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8114040 con 0x7f21d8103ac0 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8103ac0 0x7f21d819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21d6000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 0x7f21d819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8103ac0 0x7f21d819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34400/0 (socket says 172.21.15.3:34400) 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21d6a00640 1 -- 172.21.15.3:0/2304464569 learned_addr learned my addr 172.21.15.3:0/2304464569 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.463+0000 7f21d7400640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8109ac0 0x7f21d819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21d6000640 1 -- 172.21.15.3:0/2304464569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8103ac0 msgr2=0x7f21d819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21d6000640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8103ac0 0x7f21d819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21d6000640 1 -- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8109ac0 msgr2=0x7f21d819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21d6000640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8109ac0 0x7f21d819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21d6000640 1 -- 172.21.15.3:0/2304464569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d819f390 con 0x7f21d8104460 2024-08-01T04:26:45.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21d6000640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 0x7f21d819a920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f21c4005940 tx=0x7f21c4031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:45.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.464+0000 7f21cf400640 1 -- 172.21.15.3:0/2304464569 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c403f070 con 0x7f21d8104460 2024-08-01T04:26:45.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.465+0000 7f21cf400640 1 -- 172.21.15.3:0/2304464569 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c4003ad0 con 0x7f21d8104460 2024-08-01T04:26:45.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.465+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d819f620 con 0x7f21d8104460 2024-08-01T04:26:45.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.465+0000 7f21cf400640 1 -- 172.21.15.3:0/2304464569 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4035070 con 0x7f21d8104460 2024-08-01T04:26:45.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.465+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d81a6f60 con 0x7f21d8104460 2024-08-01T04:26:45.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.468+0000 7f21e0879740 1 Processor -- start 2024-08-01T04:26:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.468+0000 7f21e0879740 1 -- start start 2024-08-01T04:26:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.468+0000 7f21cf400640 1 -- 172.21.15.3:0/2304464569 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f21c40035f0 con 0x7f21d8104460 2024-08-01T04:26:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21e0879740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757d88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21e0879740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21e0879740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21e0879740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561e7573dbf0 con 0x561e757e36d0 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21e0879740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561e756d5b20 con 0x561e757d84f0 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21e0879740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e75739bb0 con 0x561e757d8e10 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757d88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757d88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34310/0 (socket says 172.21.15.3:34310) 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 -- 172.21.15.3:0/856010420 learned_addr learned my addr 172.21.15.3:0/856010420 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d6000640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d6a00640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 -- 172.21.15.3:0/856010420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 msgr2=0x561e757e3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 -- 172.21.15.3:0/856010420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 msgr2=0x561e757e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.469+0000 7f21d7400640 1 -- 172.21.15.3:0/856010420 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e756dc7f0 con 0x561e757d84f0 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21d6a00640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:45.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21d6000640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21d7400640 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757d88d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009ca0 tx=0x7f21d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54e22575428b5222 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21c3e00640 1 -- 172.21.15.3:0/856010420 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d001e070 con 0x561e757d84f0 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21c3e00640 1 -- 172.21.15.3:0/856010420 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d0009e50 con 0x561e757d84f0 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21c3e00640 1 -- 172.21.15.3:0/856010420 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d000eac0 con 0x561e757d84f0 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.470+0000 7f21cf400640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f21bc0a2cf0 0x7f21bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21cf400640 1 -- 172.21.15.3:0/2304464569 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f21c40ef980 con 0x7f21d8104460 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21d6a00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f21bc0a2cf0 0x7f21bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 -- 172.21.15.3:0/856010420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 msgr2=0x561e757d88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757d88d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009ca0 tx=0x7f21d0011810 comp rx=0 tx=0).stop 2024-08-01T04:26:45.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 -- 172.21.15.3:0/856010420 shutdown_connections 2024-08-01T04:26:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 --2- 172.21.15.3:0/856010420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757d88d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 -- 172.21.15.3:0/856010420 >> 172.21.15.3:0/856010420 conn(0x561e757d4a80 msgr2=0x561e757d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:45.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 -- 172.21.15.3:0/856010420 shutdown_connections 2024-08-01T04:26:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.471+0000 7f21e0879740 1 -- 172.21.15.3:0/856010420 wait complete. 2024-08-01T04:26:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.472+0000 7f21d6a00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f21bc0a2cf0 0x7f21bc0a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009fd0 tx=0x7f21c8009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.472+0000 7f21e0879740 1 -- start start 2024-08-01T04:26:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.472+0000 7f21e0879740 1 Processor -- start 2024-08-01T04:26:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757e0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561e757e62a0 con 0x561e757e36d0 2024-08-01T04:26:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x561e756f8cf0 con 0x561e757d84f0 2024-08-01T04:26:45.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x561e757e6420 con 0x561e757d8e10 2024-08-01T04:26:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21e0879740 10 client.0 _ll_register_callbacks cb 0x561e756e4e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21d7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21d6000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21d7400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46970/0 (socket says 172.21.15.3:46970) 2024-08-01T04:26:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21d7400640 1 -- 172.21.15.3:0/3544105005 learned_addr learned my addr 172.21.15.3:0/3544105005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.473+0000 7f21d6000640 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34418/0 (socket says 172.21.15.3:34418) 2024-08-01T04:26:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21d6a00640 1 -- 172.21.15.3:0/3544105005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 msgr2=0x561e757e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21d6a00640 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21d6a00640 1 -- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 msgr2=0x561e757e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21d6a00640 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21d6a00640 1 -- 172.21.15.3:0/3544105005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e754af3c0 con 0x561e757d84f0 2024-08-01T04:26:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21d6a00640 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757e0640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f21c80049d0 tx=0x7f21c8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb278e0b5742d3b server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:26:45.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.474+0000 7f21c0c00640 1 -- 172.21.15.3:0/3544105005 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c802d060 con 0x561e757d84f0 2024-08-01T04:26:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.475+0000 7f21c0c00640 1 -- 172.21.15.3:0/3544105005 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f21c8017030 con 0x561e757d84f0 2024-08-01T04:26:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.475+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d0009e50 con 0x561e757d84f0 2024-08-01T04:26:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.475+0000 7f21c0c00640 1 -- 172.21.15.3:0/3544105005 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21c801d560 con 0x561e757d84f0 2024-08-01T04:26:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.475+0000 7f21e0879740 10 client.26528 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:45.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.475+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e757f2030 con 0x561e757d84f0 2024-08-01T04:26:45.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.475+0000 7f21c0c00640 1 -- 172.21.15.3:0/3544105005 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d0009e50 con 0x561e757d84f0 2024-08-01T04:26:45.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21c0c00640 1 -- 172.21.15.3:0/3544105005 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f21c80289f0 con 0x561e757d84f0 2024-08-01T04:26:45.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21e0879740 10 client.26528 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:45.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21e0879740 10 client.26528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:26:45.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21e0879740 20 client.26528 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:45.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21e0879740 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e757f34f0 0x561e757f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21e0879740 4 client.26528 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:45.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e7574b160 con 0x561e757f34f0 2024-08-01T04:26:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.476+0000 7f21d6000640 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e757f34f0 0x561e757f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.477+0000 7f21d6000640 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e757f34f0 0x561e757f58e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21fe02aacecaaaf3 server_cookie=9f3c8afa02a1d17 in_seq=0 out_seq=0 2024-08-01T04:26:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.477+0000 7f21c0c00640 10 client.26528 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.477+0000 7f21c0c00640 1 -- 172.21.15.3:0/3544105005 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e7574b160 con 0x561e757f34f0 2024-08-01T04:26:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.477+0000 7f21c0c00640 10 client.26528 handle_command_reply: tid=0 2024-08-01T04:26:45.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 client.26528 shutdown 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 msgr2=0x561e757e0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757e0640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f21c80049d0 tx=0x7f21c8004ab0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 shutdown_connections 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e757f34f0 0x561e757f58e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757e36d0 0x561e757e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757d8e10 0x561e757e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 --2- 172.21.15.3:0/3544105005 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757e0640 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21c0c00640 0 client.26528 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:45.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 >> 172.21.15.3:0/3544105005 conn(0x561e757d4a80 msgr2=0x561e757d3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:45.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 shutdown_connections 2024-08-01T04:26:45.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 1 -- 172.21.15.3:0/3544105005 wait complete. 2024-08-01T04:26:45.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.478+0000 7f21e0879740 20 client.26528 trim_cache size 0 max 16384 2024-08-01T04:26:45.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.493+0000 7f21e0879740 1 Processor -- start 2024-08-01T04:26:45.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.493+0000 7f21e0879740 1 -- start start 2024-08-01T04:26:45.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.493+0000 7f21e0879740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 0x561e757cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21e0879740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d8e10 0x561e757daf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21e0879740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21e0879740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81b2310 con 0x561e757d84f0 2024-08-01T04:26:45.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21e0879740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81afbe0 con 0x561e757d8e10 2024-08-01T04:26:45.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21e0879740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81ada50 con 0x561e757e36d0 2024-08-01T04:26:45.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 0x561e757cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d8e10 0x561e757daf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 0x561e757cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46972/0 (socket says 172.21.15.3:46972) 2024-08-01T04:26:45.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6a00640 1 -- 172.21.15.3:0/3998560899 learned_addr learned my addr 172.21.15.3:0/3998560899 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:45.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d7400640 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6000640 1 -- 172.21.15.3:0/3998560899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 msgr2=0x561e757e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6000640 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757e25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6000640 1 -- 172.21.15.3:0/3998560899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 msgr2=0x561e757cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6000640 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 0x561e757cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6000640 1 -- 172.21.15.3:0/3998560899 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e757e3190 con 0x561e757d8e10 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.494+0000 7f21d6a00640 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 0x561e757cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.495+0000 7f21d6000640 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d8e10 0x561e757daf50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21c40ed000 tx=0x7f21c402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9baa848b4531c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.495+0000 7f21c3400640 1 -- 172.21.15.3:0/3998560899 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4048040 con 0x561e757d8e10 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.495+0000 7f21c3400640 1 -- 172.21.15.3:0/3998560899 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c40c0180 con 0x561e757d8e10 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.495+0000 7f21c3400640 1 -- 172.21.15.3:0/3998560899 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40b84f0 con 0x561e757d8e10 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 -- 172.21.15.3:0/3998560899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d8e10 msgr2=0x561e757daf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d8e10 0x561e757daf50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21c40ed000 tx=0x7f21c402a420 comp rx=0 tx=0).stop 2024-08-01T04:26:45.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 -- 172.21.15.3:0/3998560899 shutdown_connections 2024-08-01T04:26:45.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d8e10 0x561e757daf50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 --2- 172.21.15.3:0/3998560899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d84f0 0x561e757cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 -- 172.21.15.3:0/3998560899 >> 172.21.15.3:0/3998560899 conn(0x561e757d4a80 msgr2=0x561e757e1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:45.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 -- 172.21.15.3:0/3998560899 shutdown_connections 2024-08-01T04:26:45.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 -- 172.21.15.3:0/3998560899 wait complete. 2024-08-01T04:26:45.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.496+0000 7f21e0879740 1 -- start start 2024-08-01T04:26:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 Processor -- start 2024-08-01T04:26:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 0x561e757dd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757ddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81ae0a0 con 0x561e757d8e10 2024-08-01T04:26:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81af510 con 0x561e757d84f0 2024-08-01T04:26:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81ae770 con 0x561e757e36d0 2024-08-01T04:26:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21e0879740 10 client.0 _ll_register_callbacks cb 0x7f21c800f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:45.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21d6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 0x561e757dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.497+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 0x561e757dd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46976/0 (socket says 172.21.15.3:46976) 2024-08-01T04:26:45.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6000640 1 -- 172.21.15.3:0/4019498363 learned_addr learned my addr 172.21.15.3:0/4019498363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d7400640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757ddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6a00640 1 -- 172.21.15.3:0/4019498363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 msgr2=0x561e757ddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6a00640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757ddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6a00640 1 -- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 msgr2=0x561e757dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6a00640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 0x561e757dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6a00640 1 -- 172.21.15.3:0/4019498363 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7585d160 con 0x561e757d84f0 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6000640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 0x561e757dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21d6a00640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757ebf30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f21c8004a80 tx=0x7f21c801d270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=790384b3c7ee3c68 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.498+0000 7f21c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-08-01T04:26:45.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21c0c00640 1 -- 172.21.15.3:0/4019498363 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c801b9d0 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21c40c0180 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21c0c00640 1 -- 172.21.15.3:0/4019498363 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f21c801bb70 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21c0c00640 1 -- 172.21.15.3:0/4019498363 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21c801bd70 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21e0879740 10 client.26534 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e7585e320 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21c0c00640 1 -- 172.21.15.3:0/4019498363 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21c40c0180 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.499+0000 7f21c0c00640 1 -- 172.21.15.3:0/4019498363 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f21c8028620 con 0x561e757d84f0 2024-08-01T04:26:45.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21e0879740 10 client.26534 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21e0879740 10 client.26534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-08-01T04:26:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21e0879740 20 client.26534 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21e0879740 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e7585eaa0 0x561e75860ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21e0879740 4 client.26534 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:26:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 --> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e75745670 con 0x561e7585eaa0 2024-08-01T04:26:45.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21d6000640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e7585eaa0 0x561e75860ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21d6000640 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e7585eaa0 0x561e75860ec0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97dc8aefc0db60b1 server_cookie=b7d07ba7d18d804d in_seq=0 out_seq=0 2024-08-01T04:26:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.500+0000 7f21c0c00640 10 client.26534 ms_handle_connect on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21c0c00640 1 -- 172.21.15.3:0/4019498363 <== mds.1 v2:172.21.15.3:6840/2435488646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e75745670 con 0x561e7585eaa0 2024-08-01T04:26:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21c0c00640 10 client.26534 handle_command_reply: tid=0 2024-08-01T04:26:45.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 client.26534 shutdown 2024-08-01T04:26:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 msgr2=0x561e757ebf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757ebf30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f21c8004a80 tx=0x7f21c801d270 comp rx=0 tx=0).stop 2024-08-01T04:26:45.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 shutdown_connections 2024-08-01T04:26:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:6840/2435488646,v1:172.21.15.3:6841/2435488646] conn(0x561e7585eaa0 0x561e75860ec0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x561e757e36d0 0x561e757ddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561e757d8e10 0x561e757dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 --2- 172.21.15.3:0/4019498363 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e757d84f0 0x561e757ebf30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 >> 172.21.15.3:0/4019498363 conn(0x561e757d4a80 msgr2=0x561e757d3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21c0c00640 0 client.26534 ms_handle_reset on v2:172.21.15.3:6840/2435488646 2024-08-01T04:26:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 shutdown_connections 2024-08-01T04:26:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 1 -- 172.21.15.3:0/4019498363 wait complete. 2024-08-01T04:26:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.501+0000 7f21e0879740 20 client.26534 trim_cache size 0 max 16384 2024-08-01T04:26:45.519 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:26:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f21bc0a2cf0 msgr2=0x7f21bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f21bc0a2cf0 0x7f21bc0a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009fd0 tx=0x7f21c8009500 comp rx=0 tx=0).stop 2024-08-01T04:26:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 msgr2=0x7f21d819a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 0x7f21d819a920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f21c4005940 tx=0x7f21c4031b50 comp rx=0 tx=0).stop 2024-08-01T04:26:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 shutdown_connections 2024-08-01T04:26:45.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d8109ac0 0x7f21d819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21d8104460 0x7f21d819a920 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f21bc0a2cf0 0x7f21bc0a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 --2- 172.21.15.3:0/2304464569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21d8103ac0 0x7f21d819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:45.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.502+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 >> 172.21.15.3:0/2304464569 conn(0x7f21d80ff7c0 msgr2=0x7f21d81000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:45.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.503+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 shutdown_connections 2024-08-01T04:26:45.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:45.503+0000 7f21ddc00640 1 -- 172.21.15.3:0/2304464569 wait complete. 2024-08-01T04:26:46.012 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:45 smithi003 ceph-mon[32262]: pgmap v325: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:45 smithi160 ceph-mon[37224]: pgmap v325: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:46.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:45 smithi163 ceph-mon[37383]: pgmap v325: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:46.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-01T04:26:46.987 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:47.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 -- 172.21.15.3:0/3948388570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981038e0 msgr2=0x7f4398103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 --2- 172.21.15.3:0/3948388570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981038e0 0x7f4398103d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4384009930 tx=0x7f438402a420 comp rx=0 tx=0).stop 2024-08-01T04:26:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 -- 172.21.15.3:0/3948388570 shutdown_connections 2024-08-01T04:26:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 --2- 172.21.15.3:0/3948388570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104280 0x7f43981107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 --2- 172.21.15.3:0/3948388570 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981038e0 0x7f4398103d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 --2- 172.21.15.3:0/3948388570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981098e0 0x7f4398109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 -- 172.21.15.3:0/3948388570 >> 172.21.15.3:0/3948388570 conn(0x7f43980ff5a0 msgr2=0x7f43981019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:47.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 -- 172.21.15.3:0/3948388570 shutdown_connections 2024-08-01T04:26:47.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.688+0000 7f439da00640 1 -- 172.21.15.3:0/3948388570 wait complete. 2024-08-01T04:26:47.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.689+0000 7f439da00640 1 Processor -- start 2024-08-01T04:26:47.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.689+0000 7f439da00640 1 -- start start 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439da00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 0x7f439819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439da00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104280 0x7f439819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439da00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981098e0 0x7f439819eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439da00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4398113d30 con 0x7f43981038e0 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439da00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4398113bb0 con 0x7f4398104280 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439da00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4398113eb0 con 0x7f43981098e0 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f4397e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 0x7f439819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f4397e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 0x7f439819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46990/0 (socket says 172.21.15.3:46990) 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f4397e00640 1 -- 172.21.15.3:0/2298631321 learned_addr learned my addr 172.21.15.3:0/2298631321 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f439d000640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981098e0 0x7f439819eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.690+0000 7f4397400640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104280 0x7f439819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4397e00640 1 -- 172.21.15.3:0/2298631321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981098e0 msgr2=0x7f439819eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4397e00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981098e0 0x7f439819eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4397e00640 1 -- 172.21.15.3:0/2298631321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104280 msgr2=0x7f439819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4397e00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104280 0x7f439819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4397e00640 1 -- 172.21.15.3:0/2298631321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439819f180 con 0x7f43981038e0 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4397e00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 0x7f439819a1d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f438800e9f0 tx=0x7f438800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4394c00640 1 -- 172.21.15.3:0/2298631321 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438800cd80 con 0x7f43981038e0 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4394c00640 1 -- 172.21.15.3:0/2298631321 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4388010910 con 0x7f43981038e0 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f4394c00640 1 -- 172.21.15.3:0/2298631321 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388018990 con 0x7f43981038e0 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.691+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439819f470 con 0x7f43981038e0 2024-08-01T04:26:47.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.692+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981a6d50 con 0x7f43981038e0 2024-08-01T04:26:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.695+0000 7f43a1acc740 1 Processor -- start 2024-08-01T04:26:47.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.695+0000 7f4394c00640 1 -- 172.21.15.3:0/2298631321 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4388010ac0 con 0x7f43981038e0 2024-08-01T04:26:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.695+0000 7f43a1acc740 1 -- start start 2024-08-01T04:26:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce55fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce56a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f43a1acc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e9ce4c4bf0 con 0x55e9ce56a8c0 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f43a1acc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e9ce45cb20 con 0x55e9ce55f6e0 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f43a1acc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e9ce4c0bb0 con 0x55e9ce560000 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f4397e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce56a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f4397e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce56a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34470/0 (socket says 172.21.15.3:34470) 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f4397e00640 1 -- 172.21.15.3:0/2321304167 learned_addr learned my addr 172.21.15.3:0/2321304167 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:47.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f4397400640 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.696+0000 7f439d000640 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce55fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4397e00640 1 -- 172.21.15.3:0/2321304167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 msgr2=0x55e9ce55fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4397e00640 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce55fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4397e00640 1 -- 172.21.15.3:0/2321304167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 msgr2=0x55e9ce56ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4397e00640 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4397e00640 1 -- 172.21.15.3:0/2321304167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ce4637f0 con 0x55e9ce560000 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f439d000640 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce55fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4397e00640 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce56a380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f43880cccb0 tx=0x7f43880d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a569d53f78b9d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:47.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.697+0000 7f4394c00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f437c0a2d70 0x7f437c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.698+0000 7f4394c00640 1 -- 172.21.15.3:0/2298631321 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f43880cf6f0 con 0x7f43981038e0 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.698+0000 7f439d000640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f437c0a2d70 0x7f437c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.698+0000 7f438d600640 1 -- 172.21.15.3:0/2321304167 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880de070 con 0x55e9ce560000 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.698+0000 7f438d600640 1 -- 172.21.15.3:0/2321304167 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438809bbf0 con 0x55e9ce560000 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.698+0000 7f438d600640 1 -- 172.21.15.3:0/2321304167 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438809b730 con 0x55e9ce560000 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 -- 172.21.15.3:0/2321304167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 msgr2=0x55e9ce56a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f439d000640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f437c0a2d70 0x7f437c0a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4390009fd0 tx=0x7f4390009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:47.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce56a380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f43880cccb0 tx=0x7f43880d7670 comp rx=0 tx=0).stop 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 -- 172.21.15.3:0/2321304167 shutdown_connections 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce56a380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 --2- 172.21.15.3:0/2321304167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce55fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 -- 172.21.15.3:0/2321304167 >> 172.21.15.3:0/2321304167 conn(0x55e9ce55bc70 msgr2=0x55e9ce55c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 -- 172.21.15.3:0/2321304167 shutdown_connections 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.699+0000 7f43a1acc740 1 -- 172.21.15.3:0/2321304167 wait complete. 2024-08-01T04:26:47.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 -- start start 2024-08-01T04:26:47.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 Processor -- start 2024-08-01T04:26:47.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 0x55e9ce567850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce560000 0x55e9ce567d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce56a8c0 0x55e9ce5682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e9ce56d490 con 0x55e9ce560000 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55e9ce47fcf0 con 0x55e9ce56a8c0 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.700+0000 7f43a1acc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x55e9ce56d610 con 0x55e9ce55f6e0 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f43a1acc740 10 client.0 _ll_register_callbacks cb 0x55e9ce46be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 0x55e9ce567850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 0x55e9ce567850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34476/0 (socket says 172.21.15.3:34476) 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 -- 172.21.15.3:0/3141027686 learned_addr learned my addr 172.21.15.3:0/3141027686 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397400640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce560000 0x55e9ce567d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f439d000640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce56a8c0 0x55e9ce5682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 -- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce56a8c0 msgr2=0x55e9ce5682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce56a8c0 0x55e9ce5682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 -- 172.21.15.3:0/3141027686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce560000 msgr2=0x55e9ce567d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce560000 0x55e9ce567d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.701+0000 7f4397e00640 1 -- 172.21.15.3:0/3141027686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ce2363c0 con 0x55e9ce55f6e0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f4397e00640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 0x55e9ce567850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4388000c00 tx=0x7f438800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2d9002a1dde8d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f4382000640 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f4382000640 1 -- 172.21.15.3:0/3141027686 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880de070 con 0x55e9ce55f6e0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f4382000640 1 -- 172.21.15.3:0/3141027686 <== mon.1 v2:172.21.15.163:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f43880ccff0 con 0x55e9ce55f6e0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438809bbf0 con 0x55e9ce55f6e0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f4382000640 1 -- 172.21.15.3:0/3141027686 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f438809b550 con 0x55e9ce55f6e0 2024-08-01T04:26:47.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f43a1acc740 10 client.26518 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.702+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9ce56d490 con 0x55e9ce55f6e0 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f4382000640 1 -- 172.21.15.3:0/3141027686 <== mon.1 v2:172.21.15.163:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438809bbf0 con 0x55e9ce55f6e0 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f4382000640 1 -- 172.21.15.3:0/3141027686 <== mon.1 v2:172.21.15.163:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f438809be50 con 0x55e9ce55f6e0 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f43a1acc740 10 client.26518 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f43a1acc740 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f43a1acc740 20 client.26518 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f43a1acc740 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce57a360 0x55e9ce57c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f43a1acc740 4 client.26518 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-01T04:26:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ce4d2160 con 0x55e9ce57a360 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.703+0000 7f4397400640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce57a360 0x55e9ce57c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.704+0000 7f4397400640 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce57a360 0x55e9ce57c750 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8bdef24184a5db0 server_cookie=1e9e2a27c94b089f in_seq=0 out_seq=0 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.704+0000 7f4382000640 10 client.26518 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f4382000640 1 -- 172.21.15.3:0/3141027686 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e9ce4d2160 con 0x55e9ce57a360 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f4382000640 10 client.26518 handle_command_reply: tid=0 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 client.26518 shutdown 2024-08-01T04:26:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 msgr2=0x55e9ce567850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 0x55e9ce567850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4388000c00 tx=0x7f438800cf90 comp rx=0 tx=0).stop 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 shutdown_connections 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce57a360 0x55e9ce57c750 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce56a8c0 0x55e9ce5682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce560000 0x55e9ce567d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 --2- 172.21.15.3:0/3141027686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce55f6e0 0x55e9ce567850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f4382000640 0 client.26518 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 >> 172.21.15.3:0/3141027686 conn(0x55e9ce55bc70 msgr2=0x55e9ce561950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 shutdown_connections 2024-08-01T04:26:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.705+0000 7f43a1acc740 1 -- 172.21.15.3:0/3141027686 wait complete. 2024-08-01T04:26:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.706+0000 7f43a1acc740 20 client.26518 trim_cache size 0 max 16384 2024-08-01T04:26:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.720+0000 7f43a1acc740 1 Processor -- start 2024-08-01T04:26:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.720+0000 7f43a1acc740 1 -- start start 2024-08-01T04:26:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce562140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce566120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce568120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f43a1acc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f43981b20d0 con 0x55e9ce56a8c0 2024-08-01T04:26:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f43a1acc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f43981af9a0 con 0x55e9ce55f6e0 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f43a1acc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43981ad810 con 0x55e9ce560000 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f439d000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce568120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f4397e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce562140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f4397400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce566120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f4397e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce562140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34384/0 (socket says 172.21.15.3:34384) 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f4397400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce566120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34478/0 (socket says 172.21.15.3:34478) 2024-08-01T04:26:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.721+0000 7f4397e00640 1 -- 172.21.15.3:0/937878472 learned_addr learned my addr 172.21.15.3:0/937878472 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397400640 1 -- 172.21.15.3:0/937878472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 msgr2=0x55e9ce562140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397400640 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce562140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397400640 1 -- 172.21.15.3:0/937878472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 msgr2=0x55e9ce568120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397400640 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce568120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397400640 1 -- 172.21.15.3:0/937878472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ce560400 con 0x55e9ce560000 2024-08-01T04:26:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397e00640 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce562140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f4397400640 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce566120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f438402a940 tx=0x7f4384000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed7c0dc5a0ebd153 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.722+0000 7f438cc00640 1 -- 172.21.15.3:0/937878472 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384035040 con 0x55e9ce560000 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f438cc00640 1 -- 172.21.15.3:0/937878472 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438404b070 con 0x55e9ce560000 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f438cc00640 1 -- 172.21.15.3:0/937878472 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384031810 con 0x55e9ce560000 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 -- 172.21.15.3:0/937878472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 msgr2=0x55e9ce566120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce566120 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f438402a940 tx=0x7f4384000bc0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 -- 172.21.15.3:0/937878472 shutdown_connections 2024-08-01T04:26:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce56a8c0 0x55e9ce568120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce560000 0x55e9ce566120 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 --2- 172.21.15.3:0/937878472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce55f6e0 0x55e9ce562140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 -- 172.21.15.3:0/937878472 >> 172.21.15.3:0/937878472 conn(0x55e9ce55bc70 msgr2=0x55e9ce5698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 -- 172.21.15.3:0/937878472 shutdown_connections 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.723+0000 7f43a1acc740 1 -- 172.21.15.3:0/937878472 wait complete. 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.724+0000 7f43a1acc740 1 -- start start 2024-08-01T04:26:47.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.724+0000 7f43a1acc740 1 Processor -- start 2024-08-01T04:26:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 0x55e9ce572b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce560000 0x55e9ce5730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f43981ade60 con 0x55e9ce55f6e0 2024-08-01T04:26:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f43981af2d0 con 0x55e9ce560000 2024-08-01T04:26:47.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f43981ae530 con 0x55e9ce56a8c0 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f43a1acc740 10 client.0 _ll_register_callbacks cb 0x7f43880d5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 0x55e9ce572b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce560000 0x55e9ce5730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 0x55e9ce572b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47006/0 (socket says 172.21.15.3:47006) 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397e00640 1 -- 172.21.15.3:0/1511156131 learned_addr learned my addr 172.21.15.3:0/1511156131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce560000 0x55e9ce5730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34392/0 (socket says 172.21.15.3:34392) 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f439d000640 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397e00640 1 -- 172.21.15.3:0/1511156131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce56a8c0 msgr2=0x55e9ce56df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.725+0000 7f4397e00640 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4397e00640 1 -- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce560000 msgr2=0x55e9ce5730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4397e00640 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce560000 0x55e9ce5730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4397e00640 1 -- 172.21.15.3:0/1511156131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ce56e790 con 0x55e9ce55f6e0 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4397e00640 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 0x55e9ce572b60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4388000c00 tx=0x7f438800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aae91c4eb104bbe server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4382000640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4382000640 1 -- 172.21.15.3:0/1511156131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880de070 con 0x55e9ce55f6e0 2024-08-01T04:26:47.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f4382000640 1 -- 172.21.15.3:0/1511156131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f4388018b30 con 0x55e9ce55f6e0 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.726+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438404b070 con 0x55e9ce55f6e0 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f4382000640 1 -- 172.21.15.3:0/1511156131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4388097e90 con 0x55e9ce55f6e0 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9ce5e4920 con 0x55e9ce55f6e0 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f4382000640 1 -- 172.21.15.3:0/1511156131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438404b070 con 0x55e9ce55f6e0 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f4382000640 1 -- 172.21.15.3:0/1511156131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f43880ba3d0 con 0x55e9ce55f6e0 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-08-01T04:26:47.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 20 client.16368 populate_metadata read hostname 'smithi003' 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce5e4d50 0x55e9ce5e7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 4 client.16368 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.727+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 --> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9ce4cc670 con 0x55e9ce5e4d50 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.728+0000 7f4397400640 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce5e4d50 0x55e9ce5e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.728+0000 7f4397400640 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce5e4d50 0x55e9ce5e7170 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676b7f54673cbe41 server_cookie=e27b0164e9488eff in_seq=0 out_seq=0 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.728+0000 7f4382000640 10 client.16368 ms_handle_connect on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:47.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.728+0000 7f4382000640 1 -- 172.21.15.3:0/1511156131 <== mds.0 v2:172.21.15.160:6838/1741438883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9ce4cc670 con 0x55e9ce5e4d50 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.728+0000 7f4382000640 10 client.16368 handle_command_reply: tid=0 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.728+0000 7f43a1acc740 1 client.16368 shutdown 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 msgr2=0x55e9ce572b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 0x55e9ce572b60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4388000c00 tx=0x7f438800cf20 comp rx=0 tx=0).stop 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 shutdown_connections 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:6838/1741438883,v1:172.21.15.160:6839/1741438883] conn(0x55e9ce5e4d50 0x55e9ce5e7170 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55e9ce56a8c0 0x55e9ce56df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e9ce560000 0x55e9ce5730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 --2- 172.21.15.3:0/1511156131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e9ce55f6e0 0x55e9ce572b60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 >> 172.21.15.3:0/1511156131 conn(0x55e9ce55bc70 msgr2=0x55e9ce55a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f4382000640 0 client.16368 ms_handle_reset on v2:172.21.15.160:6838/1741438883 2024-08-01T04:26:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 shutdown_connections 2024-08-01T04:26:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 1 -- 172.21.15.3:0/1511156131 wait complete. 2024-08-01T04:26:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.729+0000 7f43a1acc740 20 client.16368 trim_cache size 0 max 16384 2024-08-01T04:26:47.738 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-08-01T04:26:47.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f437c0a2d70 msgr2=0x7f437c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f437c0a2d70 0x7f437c0a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4390009fd0 tx=0x7f4390009500 comp rx=0 tx=0).stop 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 msgr2=0x7f439819a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 0x7f439819a1d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f438800e9f0 tx=0x7f438800eeb0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 shutdown_connections 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f437c0a2d70 0x7f437c0a5230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43981098e0 0x7f439819eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4398104280 0x7f439819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 --2- 172.21.15.3:0/2298631321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43981038e0 0x7f439819a1d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 >> 172.21.15.3:0/2298631321 conn(0x7f43980ff5a0 msgr2=0x7f43980ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.730+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 shutdown_connections 2024-08-01T04:26:47.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:47.731+0000 7f439da00640 1 -- 172.21.15.3:0/2298631321 wait complete. 2024-08-01T04:26:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:47 smithi003 ceph-mon[32262]: pgmap v326: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:48.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:47 smithi160 ceph-mon[37224]: pgmap v326: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:48.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:47 smithi163 ceph-mon[37383]: pgmap v326: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:48.661 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_d99c9769be1521395979e8c23d9073adbf1ff43f/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-01T04:26:48.666 INFO:tasks.fwd_scrub:done joining 2024-08-01T04:26:48.667 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-01T04:26:48.678 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-01T04:26:48.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 -- 172.21.15.3:0/3770196347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf81041a0 msgr2=0x7feaf81106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:48.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 --2- 172.21.15.3:0/3770196347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf81041a0 0x7feaf81106d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009c10 tx=0x7feaf002a6f0 comp rx=0 tx=0).stop 2024-08-01T04:26:48.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 -- 172.21.15.3:0/3770196347 shutdown_connections 2024-08-01T04:26:48.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 --2- 172.21.15.3:0/3770196347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf81041a0 0x7feaf81106d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:48.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 --2- 172.21.15.3:0/3770196347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf8103800 0x7feaf8103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:48.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 --2- 172.21.15.3:0/3770196347 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf8109810 0x7feaf8109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:48.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.908+0000 7feaf7e00640 1 -- 172.21.15.3:0/3770196347 >> 172.21.15.3:0/3770196347 conn(0x7feaf80ff500 msgr2=0x7feaf8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:48.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.909+0000 7feaf7e00640 1 -- 172.21.15.3:0/3770196347 shutdown_connections 2024-08-01T04:26:48.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.909+0000 7feaf7e00640 1 -- 172.21.15.3:0/3770196347 wait complete. 2024-08-01T04:26:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 Processor -- start 2024-08-01T04:26:48.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 -- start start 2024-08-01T04:26:48.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf8103800 0x7feaf819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:48.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 0x7feaf819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:48.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 0x7feaf819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:48.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8113c50 con 0x7feaf81041a0 2024-08-01T04:26:48.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8113ad0 con 0x7feaf8103800 2024-08-01T04:26:48.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8113dd0 con 0x7feaf8109810 2024-08-01T04:26:48.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 0x7feaf819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 0x7feaf819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaf6a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf8103800 0x7feaf819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.910+0000 7feaee000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 0x7feaf819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47038/0 (socket says 172.21.15.3:47038) 2024-08-01T04:26:48.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 0x7feaf819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34512/0 (socket says 172.21.15.3:34512) 2024-08-01T04:26:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaee000640 1 -- 172.21.15.3:0/2629024730 learned_addr learned my addr 172.21.15.3:0/2629024730 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 -- 172.21.15.3:0/2629024730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf8103800 msgr2=0x7feaf819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf8103800 0x7feaf819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:48.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 -- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 msgr2=0x7feaf819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 0x7feaf819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 -- 172.21.15.3:0/2629024730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf819f110 con 0x7feaf8109810 2024-08-01T04:26:48.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaee000640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 0x7feaf819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.911+0000 7feaf7400640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 0x7feaf819e9e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feaf0034040 tx=0x7feaf00305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.912+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0041020 con 0x7feaf8109810 2024-08-01T04:26:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.912+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf819f3a0 con 0x7feaf8109810 2024-08-01T04:26:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.912+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0003e50 con 0x7feaf8109810 2024-08-01T04:26:48.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.912+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0036040 con 0x7feaf8109810 2024-08-01T04:26:48.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.912+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a6c90 con 0x7feaf8109810 2024-08-01T04:26:48.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.913+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8103c60 con 0x7feaf8109810 2024-08-01T04:26:48.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.915+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7feaf0044020 con 0x7feaf8109810 2024-08-01T04:26:48.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.916+0000 7feafca00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feadc0a2d00 0x7feadc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.916+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7feaf00f00e0 con 0x7feaf8109810 2024-08-01T04:26:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.917+0000 7feaf6a00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feadc0a2d00 0x7feadc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.917+0000 7feaf6a00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feadc0a2d00 0x7feadc0a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feae8009b50 tx=0x7feae8005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:48.919+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feaf004b020 con 0x7feaf8109810 2024-08-01T04:26:49.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.125+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaf810bd70 con 0x7feaf8109810 2024-08-01T04:26:49.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.127+0000 7feafca00640 1 -- 172.21.15.3:0/2629024730 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23268 (secure 0 0 0) 0x7feaf00bcc30 con 0x7feaf8109810 2024-08-01T04:26:49.129 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:26:49.129 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":99,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:26:19.575439+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1348130449":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/346159818":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/4015366337":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2309230633":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:26:49.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feadc0a2d00 msgr2=0x7feadc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feadc0a2d00 0x7feadc0a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feae8009b50 tx=0x7feae8005f90 comp rx=0 tx=0).stop 2024-08-01T04:26:49.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 msgr2=0x7feaf819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 0x7feaf819e9e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feaf0034040 tx=0x7feaf00305d0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 shutdown_connections 2024-08-01T04:26:49.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7feadc0a2d00 0x7feadc0a51c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf8109810 0x7feaf819e9e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feaf81041a0 0x7feaf819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 --2- 172.21.15.3:0/2629024730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feaf8103800 0x7feaf819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.130+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 >> 172.21.15.3:0/2629024730 conn(0x7feaf80ff500 msgr2=0x7feaf80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.131+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 shutdown_connections 2024-08-01T04:26:49.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.131+0000 7feaf7e00640 1 -- 172.21.15.3:0/2629024730 wait complete. 2024-08-01T04:26:49.152 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-01T04:09:04.723203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-01T04:13:50.125660+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-01T04:13:53.246715+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-01T04:26:49.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-01T04:26:49.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 -- 172.21.15.3:0/603483861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 msgr2=0x7f4c08112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/603483861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 0x7f4c08112130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009900 tx=0x7f4bf002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 -- 172.21.15.3:0/603483861 shutdown_connections 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/603483861 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c08114a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/603483861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 0x7f4c08112130 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/603483861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 0x7f4c08076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 -- 172.21.15.3:0/603483861 >> 172.21.15.3:0/603483861 conn(0x7f4c080ff670 msgr2=0x7f4c08101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 -- 172.21.15.3:0/603483861 shutdown_connections 2024-08-01T04:26:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.356+0000 7f4c0fa00640 1 -- 172.21.15.3:0/603483861 wait complete. 2024-08-01T04:26:49.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.357+0000 7f4c0fa00640 1 Processor -- start 2024-08-01T04:26:49.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.357+0000 7f4c0fa00640 1 -- start start 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0fa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 0x7f4c081a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0fa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 0x7f4c081a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0fa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c0819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0fa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08118230 con 0x7f4c08076d70 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0fa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081180b0 con 0x7f4c08112670 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0fa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081183b0 con 0x7f4c08076450 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0da00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c0819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0d000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 0x7f4c081a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0da00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c0819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34430/0 (socket says 172.21.15.3:34430) 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c0da00640 1 -- 172.21.15.3:0/3831032116 learned_addr learned my addr 172.21.15.3:0/3831032116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.358+0000 7f4c07e00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 0x7f4c081a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0da00640 1 -- 172.21.15.3:0/3831032116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 msgr2=0x7f4c081a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0da00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 0x7f4c081a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0da00640 1 -- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 msgr2=0x7f4c081a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0da00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 0x7f4c081a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0da00640 1 -- 172.21.15.3:0/3831032116 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0819ed20 con 0x7f4c08112670 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0d000640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 0x7f4c081a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.359+0000 7f4c0da00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c0819e5a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000dac0 tx=0x7f4c0000df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.360+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00017070 con 0x7f4c08112670 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.360+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c00003e50 con 0x7f4c08112670 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.360+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0819f010 con 0x7f4c08112670 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.360+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c000106c0 con 0x7f4c08112670 2024-08-01T04:26:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.360+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081ab2b0 con 0x7f4c08112670 2024-08-01T04:26:49.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.361+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08076830 con 0x7f4c08112670 2024-08-01T04:26:49.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.363+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4c00003970 con 0x7f4c08112670 2024-08-01T04:26:49.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.364+0000 7f4c05600640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f4bec0a2d80 0x7f4bec0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.365+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f4c000ceec0 con 0x7f4c08112670 2024-08-01T04:26:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.365+0000 7f4c0d000640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f4bec0a2d80 0x7f4bec0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.365+0000 7f4c0d000640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f4bec0a2d80 0x7f4bec0a5240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009fc0 tx=0x7f4bf8005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:49.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.366+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4c0009b440 con 0x7f4c08112670 2024-08-01T04:26:49.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.571+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c0802d5e0 con 0x7f4c08112670 2024-08-01T04:26:49.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.572+0000 7f4c05600640 1 -- 172.21.15.3:0/3831032116 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f4c0009f740 con 0x7f4c08112670 2024-08-01T04:26:49.574 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-08-01T04:26:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f4bec0a2d80 msgr2=0x7f4bec0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f4bec0a2d80 0x7f4bec0a5240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009fc0 tx=0x7f4bf8005f90 comp rx=0 tx=0).stop 2024-08-01T04:26:49.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 msgr2=0x7f4c0819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c0819e5a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000dac0 tx=0x7f4c0000df80 comp rx=0 tx=0).stop 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 shutdown_connections 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c08112670 0x7f4c0819e5a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c08076d70 0x7f4c081a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f4bec0a2d80 0x7f4bec0a5240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 --2- 172.21.15.3:0/3831032116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c08076450 0x7f4c081a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.574+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 >> 172.21.15.3:0/3831032116 conn(0x7f4c080ff670 msgr2=0x7f4c08113c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.575+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 shutdown_connections 2024-08-01T04:26:49.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.575+0000 7f4c0fa00640 1 -- 172.21.15.3:0/3831032116 wait complete. 2024-08-01T04:26:49.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-01T04:26:49.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 -- 172.21.15.3:0/3486110926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 msgr2=0x7fe5fc076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 --2- 172.21.15.3:0/3486110926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc076830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009ca0 tx=0x7fe5f402a8b0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 -- 172.21.15.3:0/3486110926 shutdown_connections 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 --2- 172.21.15.3:0/3486110926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5fc112550 0x7fe5fc114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 --2- 172.21.15.3:0/3486110926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 0x7fe5fc112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 --2- 172.21.15.3:0/3486110926 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc076830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 -- 172.21.15.3:0/3486110926 >> 172.21.15.3:0/3486110926 conn(0x7fe5fc0ff570 msgr2=0x7fe5fc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:49.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 -- 172.21.15.3:0/3486110926 shutdown_connections 2024-08-01T04:26:49.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.814+0000 7fe601e00640 1 -- 172.21.15.3:0/3486110926 wait complete. 2024-08-01T04:26:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.815+0000 7fe601e00640 1 Processor -- start 2024-08-01T04:26:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 -- start start 2024-08-01T04:26:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 0x7fe5fc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5fc112550 0x7fe5fc1a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1180f0 con 0x7fe5fc112550 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc117f70 con 0x7fe5fc076450 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.816+0000 7fe601e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc118270 con 0x7fe5fc076d70 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe5fbe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 0x7fe5fc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34440/0 (socket says 172.21.15.3:34440) 2024-08-01T04:26:49.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 -- 172.21.15.3:0/2488114037 learned_addr learned my addr 172.21.15.3:0/2488114037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe601400640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5fc112550 0x7fe5fc1a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 -- 172.21.15.3:0/2488114037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 msgr2=0x7fe5fc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 0x7fe5fc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 -- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5fc112550 msgr2=0x7fe5fc1a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5fc112550 0x7fe5fc1a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 -- 172.21.15.3:0/2488114037 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc1a35f0 con 0x7fe5fc076450 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe5fbe00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 0x7fe5fc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:49.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.817+0000 7fe600a00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc19e600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4005e30 tx=0x7fe5f40309b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:49.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.818+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4004040 con 0x7fe5fc076450 2024-08-01T04:26:49.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.818+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4040070 con 0x7fe5fc076450 2024-08-01T04:26:49.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.818+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc1a3880 con 0x7fe5fc076450 2024-08-01T04:26:49.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.818+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4045c90 con 0x7fe5fc076450 2024-08-01T04:26:49.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.818+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc1ab170 con 0x7fe5fc076450 2024-08-01T04:26:49.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.819+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5fc076830 con 0x7fe5fc076450 2024-08-01T04:26:49.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.821+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe5f40041e0 con 0x7fe5fc076450 2024-08-01T04:26:49.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.822+0000 7fe5f9600640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5e00a2d80 0x7fe5e00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:49.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.823+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7fe5f40f04d0 con 0x7fe5fc076450 2024-08-01T04:26:49.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.823+0000 7fe5fbe00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5e00a2d80 0x7fe5e00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:49.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.823+0000 7fe5fbe00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5e00a2d80 0x7fe5e00a5240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc1019b0 tx=0x7fe5e400b390 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:49.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:49.826+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe5f40bd130 con 0x7fe5fc076450 2024-08-01T04:26:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.034+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5fc1ab340 con 0x7fe5fc076450 2024-08-01T04:26:50.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.035+0000 7fe5f9600640 1 -- 172.21.15.3:0/2488114037 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fe5f40bca30 con 0x7fe5fc076450 2024-08-01T04:26:50.037 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-08-01T04:26:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5e00a2d80 msgr2=0x7fe5e00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5e00a2d80 0x7fe5e00a5240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc1019b0 tx=0x7fe5e400b390 comp rx=0 tx=0).stop 2024-08-01T04:26:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 msgr2=0x7fe5fc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc19e600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4005e30 tx=0x7fe5f40309b0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 shutdown_connections 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5fc112550 0x7fe5fc1a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe5e00a2d80 0x7fe5e00a5240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5fc076d70 0x7fe5fc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 --2- 172.21.15.3:0/2488114037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5fc076450 0x7fe5fc19e600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 >> 172.21.15.3:0/2488114037 conn(0x7fe5fc0ff570 msgr2=0x7fe5fc113b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.037+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 shutdown_connections 2024-08-01T04:26:50.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.038+0000 7fe601e00640 1 -- 172.21.15.3:0/2488114037 wait complete. 2024-08-01T04:26:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:49 smithi003 ceph-mon[32262]: pgmap v327: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-01T04:26:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:49 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2629024730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:26:50.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:49 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3831032116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:26:50.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-01T04:26:50.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:49 smithi163 ceph-mon[37383]: pgmap v327: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-01T04:26:50.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:49 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2629024730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:26:50.098 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:49 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3831032116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:26:50.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:49 smithi160 ceph-mon[37224]: pgmap v327: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-08-01T04:26:50.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:49 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2629024730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:26:50.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:49 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3831032116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-01T04:26:50.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.268+0000 7fbd16e00640 1 -- 172.21.15.3:0/2205090849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd10108090 msgr2=0x7fbd10108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.268+0000 7fbd16e00640 1 --2- 172.21.15.3:0/2205090849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd10108090 0x7fbd10108470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009900 tx=0x7fbd0002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 -- 172.21.15.3:0/2205090849 shutdown_connections 2024-08-01T04:26:50.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 --2- 172.21.15.3:0/2205090849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd1010d190 0x7fbd10114ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 --2- 172.21.15.3:0/2205090849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd10108a40 0x7fbd1010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 --2- 172.21.15.3:0/2205090849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd10108090 0x7fbd10108470 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 -- 172.21.15.3:0/2205090849 >> 172.21.15.3:0/2205090849 conn(0x7fbd10077810 msgr2=0x7fbd10079c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:50.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 -- 172.21.15.3:0/2205090849 shutdown_connections 2024-08-01T04:26:50.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.269+0000 7fbd16e00640 1 -- 172.21.15.3:0/2205090849 wait complete. 2024-08-01T04:26:50.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 Processor -- start 2024-08-01T04:26:50.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 -- start start 2024-08-01T04:26:50.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 0x7fbd1019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd1010d190 0x7fbd1019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101a2fd0 0x7fbd101a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10118200 con 0x7fbd1010d190 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10118080 con 0x7fbd10108090 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd16e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10118380 con 0x7fbd101a2fd0 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 0x7fbd1019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.270+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd1010d190 0x7fbd1019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 0x7fbd1019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34464/0 (socket says 172.21.15.3:34464) 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 -- 172.21.15.3:0/1874916929 learned_addr learned my addr 172.21.15.3:0/1874916929 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd14e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101a2fd0 0x7fbd101a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 -- 172.21.15.3:0/1874916929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101a2fd0 msgr2=0x7fbd101a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101a2fd0 0x7fbd101a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 -- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd1010d190 msgr2=0x7fbd1019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd1010d190 0x7fbd1019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 -- 172.21.15.3:0/1874916929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd101a3ac0 con 0x7fbd10108090 2024-08-01T04:26:50.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd07e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd1010d190 0x7fbd1019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:50.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.271+0000 7fbd0fe00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 0x7fbd1019e710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd00004770 tx=0x7fbd00003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:50.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.272+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0003f070 con 0x7fbd10108090 2024-08-01T04:26:50.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.272+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd00031ca0 con 0x7fbd10108090 2024-08-01T04:26:50.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.272+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd101a3d50 con 0x7fbd10108090 2024-08-01T04:26:50.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.272+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00035030 con 0x7fbd10108090 2024-08-01T04:26:50.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.272+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101ab510 con 0x7fbd10108090 2024-08-01T04:26:50.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.273+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd10108470 con 0x7fbd10108090 2024-08-01T04:26:50.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.275+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fbd00042070 con 0x7fbd10108090 2024-08-01T04:26:50.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.276+0000 7fbd0d600640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbcf40a2d00 0x7fbcf40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.277+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7fbd000efae0 con 0x7fbd10108090 2024-08-01T04:26:50.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.277+0000 7fbd07e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbcf40a2d00 0x7fbcf40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.277+0000 7fbd07e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbcf40a2d00 0x7fbcf40a51c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80095f0 tx=0x7fbcf8005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:50.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.280+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbd000bc630 con 0x7fbd10108090 2024-08-01T04:26:50.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.489+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd10108a40 con 0x7fbd10108090 2024-08-01T04:26:50.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.490+0000 7fbd0d600640 1 -- 172.21.15.3:0/1874916929 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fbd000bbf30 con 0x7fbd10108090 2024-08-01T04:26:50.493 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-08-01T04:26:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbcf40a2d00 msgr2=0x7fbcf40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbcf40a2d00 0x7fbcf40a51c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80095f0 tx=0x7fbcf8005f90 comp rx=0 tx=0).stop 2024-08-01T04:26:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 msgr2=0x7fbd1019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 0x7fbd1019e710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbd00004770 tx=0x7fbd00003c10 comp rx=0 tx=0).stop 2024-08-01T04:26:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 shutdown_connections 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbcf40a2d00 0x7fbcf40a51c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd101a2fd0 0x7fbd101a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd1010d190 0x7fbd1019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 --2- 172.21.15.3:0/1874916929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd10108090 0x7fbd1019e710 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 >> 172.21.15.3:0/1874916929 conn(0x7fbd10077810 msgr2=0x7fbd10078b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 shutdown_connections 2024-08-01T04:26:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.494+0000 7fbd16e00640 1 -- 172.21.15.3:0/1874916929 wait complete. 2024-08-01T04:26:50.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-01T04:26:50.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 -- 172.21.15.3:0/1520847187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5194076d70 msgr2=0x7f5194111f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 --2- 172.21.15.3:0/1520847187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5194076d70 0x7f5194111f60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f518002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 -- 172.21.15.3:0/1520847187 shutdown_connections 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 --2- 172.21.15.3:0/1520847187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51941124a0 0x7f5194114890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 --2- 172.21.15.3:0/1520847187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5194076d70 0x7f5194111f60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 --2- 172.21.15.3:0/1520847187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076450 0x7f5194076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 -- 172.21.15.3:0/1520847187 >> 172.21.15.3:0/1520847187 conn(0x7f51940ff500 msgr2=0x7f5194101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 -- 172.21.15.3:0/1520847187 shutdown_connections 2024-08-01T04:26:50.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.727+0000 7f5198a00640 1 -- 172.21.15.3:0/1520847187 wait complete. 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.728+0000 7f5198a00640 1 Processor -- start 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.728+0000 7f5198a00640 1 -- start start 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5198a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 0x7f519419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5198a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076d70 0x7f519419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5198a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 0x7f51941a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5198a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5194118050 con 0x7f51941124a0 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5198a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5194117ed0 con 0x7f5194076450 2024-08-01T04:26:50.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5198a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f51941181d0 con 0x7f5194076d70 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5193200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 0x7f51941a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5191e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076d70 0x7f519419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5192800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 0x7f519419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5193200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 0x7f51941a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47112/0 (socket says 172.21.15.3:47112) 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5192800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 0x7f519419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:34474/0 (socket says 172.21.15.3:34474) 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5193200640 1 -- 172.21.15.3:0/1320032553 learned_addr learned my addr 172.21.15.3:0/1320032553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:26:50.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.729+0000 7f5191e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076d70 0x7f519419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:34594/0 (socket says 172.21.15.3:34594) 2024-08-01T04:26:50.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5192800640 1 -- 172.21.15.3:0/1320032553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076d70 msgr2=0x7f519419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5192800640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076d70 0x7f519419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5192800640 1 -- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 msgr2=0x7f51941a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:50.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5192800640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 0x7f51941a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:50.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5192800640 1 -- 172.21.15.3:0/1320032553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51941a3570 con 0x7f5194076450 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5193200640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 0x7f51941a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.730+0000 7f5192800640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 0x7f519419e580 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5184005fd0 tx=0x7f518400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.731+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184009c30 con 0x7f5194076450 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.731+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51941a3860 con 0x7f5194076450 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.731+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184009dd0 con 0x7f5194076450 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.731+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184010460 con 0x7f5194076450 2024-08-01T04:26:50.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.731+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51941ab0f0 con 0x7f5194076450 2024-08-01T04:26:50.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.732+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5194076830 con 0x7f5194076450 2024-08-01T04:26:50.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.734+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5184003f50 con 0x7f5194076450 2024-08-01T04:26:50.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.735+0000 7f518b400640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f51780a2d00 0x7f51780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:26:50.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.736+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8629+0+0 (secure 0 0 0) 0x7f51840cf3e0 con 0x7f5194076450 2024-08-01T04:26:50.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.736+0000 7f5191e00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f51780a2d00 0x7f51780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:26:50.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.736+0000 7f5191e00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f51780a2d00 0x7f51780a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5180000c00 tx=0x7f5180005b50 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:26:50.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.739+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5184017020 con 0x7f5194076450 2024-08-01T04:26:51.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:50.998+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f519410d660 con 0x7f5194076450 2024-08-01T04:26:51.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.009+0000 7f518b400640 1 -- 172.21.15.3:0/1320032553 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740893 (secure 0 0 0) 0x7f5184084e20 con 0x7f5194076450 2024-08-01T04:26:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:26:51.022 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 14928, 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1464189073 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1464189073 2024-08-01T04:26:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.3:6801/1464189073", 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2024-08-01T04:16:15.828894+0000", 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540701547738038271, 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2024-08-01T04:26:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 25042, 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "z", 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.034 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.035 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-08-01T04:26:51.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-08-01T04:26:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-08-01T04:26:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-08-01T04:26:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-08-01T04:26:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-08-01T04:26:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-01T04:26:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-08-01T04:26:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-08-01T04:26:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-01T04:26:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-08-01T04:26:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-08-01T04:26:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-08-01T04:26:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-08-01T04:26:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-08-01T04:26:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-08-01T04:26:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-08-01T04:26:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-08-01T04:26:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-01T04:26:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-08-01T04:26:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-01T04:26:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-08-01T04:26:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-08-01T04:26:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-01T04:26:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-08-01T04:26:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-08-01T04:26:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-01T04:26:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-08-01T04:26:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-08-01T04:26:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-01T04:26:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-08-01T04:26:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-01T04:26:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-08-01T04:26:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-08-01T04:26:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-08-01T04:26:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-01T04:26:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-08-01T04:26:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-08-01T04:26:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-01T04:26:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-08-01T04:26:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-08-01T04:26:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-08-01T04:26:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-01T04:26:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-08-01T04:26:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-08-01T04:26:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-08-01T04:26:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-01T04:26:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-08-01T04:26:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-01T04:26:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-08-01T04:26:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-08-01T04:26:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-08-01T04:26:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-08-01T04:26:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-08-01T04:26:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-01T04:26:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-08-01T04:26:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-08-01T04:26:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-08-01T04:26:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-08-01T04:26:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-08-01T04:26:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-08-01T04:26:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-08-01T04:26:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-01T04:26:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-08-01T04:26:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-08-01T04:26:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-08-01T04:26:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-08-01T04:26:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-01T04:26:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-08-01T04:26:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-01T04:26:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-08-01T04:26:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-08-01T04:26:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-01T04:26:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-01T04:26:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-01T04:26:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-01T04:26:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-01T04:26:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-01T04:26:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-01T04:26:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-08-01T04:26:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-01T04:26:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-01T04:26:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-08-01T04:26:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-01T04:26:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-01T04:26:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-01T04:26:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-01T04:26:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-01T04:26:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-01T04:26:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-01T04:26:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-01T04:26:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-01T04:26:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-01T04:26:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-01T04:26:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-08-01T04:26:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-08-01T04:26:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-08-01T04:26:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-08-01T04:26:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-08-01T04:26:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-08-01T04:26:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-08-01T04:26:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-08-01T04:26:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-08-01T04:26:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-01T04:26:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-08-01T04:26:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-08-01T04:26:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-08-01T04:26:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-08-01T04:26:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-08-01T04:26:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-08-01T04:26:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-08-01T04:26:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-08-01T04:26:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-01T04:26:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-08-01T04:26:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-08-01T04:26:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-01T04:26:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-08-01T04:26:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-08-01T04:26:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-08-01T04:26:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-08-01T04:26:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-08-01T04:26:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-08-01T04:26:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-01T04:26:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-08-01T04:26:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-08-01T04:26:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-08-01T04:26:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-08-01T04:26:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-08-01T04:26:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-08-01T04:26:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-08-01T04:26:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-01T04:26:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-08-01T04:26:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-08-01T04:26:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-08-01T04:26:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-08-01T04:26:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-08-01T04:26:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-08-01T04:26:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-08-01T04:26:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-08-01T04:26:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-08-01T04:26:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-08-01T04:26:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-08-01T04:26:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-08-01T04:26:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-08-01T04:26:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-08-01T04:26:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-08-01T04:26:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-08-01T04:26:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-08-01T04:26:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-08-01T04:26:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-01T04:26:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-01T04:26:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-08-01T04:26:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-08-01T04:26:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-08-01T04:26:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-08-01T04:26:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-08-01T04:26:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-08-01T04:26:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 25061, 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-08-01T04:26:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-01T04:26:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-08-01T04:26:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-01T04:26:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-08-01T04:26:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-01T04:26:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-08-01T04:26:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-08-01T04:26:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-08-01T04:26:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-08-01T04:26:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-01T04:26:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-01T04:26:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-01T04:26:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-08-01T04:26:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-08-01T04:26:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-08-01T04:26:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-08-01T04:26:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-08-01T04:26:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-01T04:26:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-08-01T04:26:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-01T04:26:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-08-01T04:26:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-01T04:26:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-08-01T04:26:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-08-01T04:26:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-08-01T04:26:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-01T04:26:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-01T04:26:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-08-01T04:26:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-08-01T04:26:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-08-01T04:26:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-08-01T04:26:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-08-01T04:26:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-08-01T04:26:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-08-01T04:26:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-01T04:26:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-08-01T04:26:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-08-01T04:26:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-01T04:26:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-08-01T04:26:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-08-01T04:26:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-01T04:26:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-01T04:26:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-01T04:26:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-01T04:26:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-08-01T04:26:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-08-01T04:26:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-01T04:26:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-01T04:26:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-08-01T04:26:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-01T04:26:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-01T04:26:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-08-01T04:26:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-08-01T04:26:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-08-01T04:26:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-08-01T04:26:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-08-01T04:26:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-08-01T04:26:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-08-01T04:26:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-08-01T04:26:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-08-01T04:26:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-08-01T04:26:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-08-01T04:26:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-08-01T04:26:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-08-01T04:26:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-01T04:26:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-08-01T04:26:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-01T04:26:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-08-01T04:26:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-01T04:26:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-08-01T04:26:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-08-01T04:26:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-08-01T04:26:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-08-01T04:26:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-08-01T04:26:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-08-01T04:26:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-08-01T04:26:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-08-01T04:26:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-08-01T04:26:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-08-01T04:26:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-01T04:26:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-08-01T04:26:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-08-01T04:26:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-08-01T04:26:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-01T04:26:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-08-01T04:26:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-08-01T04:26:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-01T04:26:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-01T04:26:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-08-01T04:26:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-08-01T04:26:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-08-01T04:26:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-01T04:26:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-08-01T04:26:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-01T04:26:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-08-01T04:26:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-08-01T04:26:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-08-01T04:26:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-08-01T04:26:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-08-01T04:26:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-08-01T04:26:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-08-01T04:26:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-08-01T04:26:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-08-01T04:26:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-08-01T04:26:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-08-01T04:26:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-08-01T04:26:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-08-01T04:26:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-08-01T04:26:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-08-01T04:26:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-08-01T04:26:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-08-01T04:26:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-08-01T04:26:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-08-01T04:26:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-08-01T04:26:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-08-01T04:26:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-08-01T04:26:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-01T04:26:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-08-01T04:26:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-08-01T04:26:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-08-01T04:26:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-08-01T04:26:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-08-01T04:26:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-08-01T04:26:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-08-01T04:26:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-08-01T04:26:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-08-01T04:26:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.524 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.524 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.526 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-08-01T04:26:52.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.530 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-08-01T04:26:52.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.530 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.531 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.532 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.537 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.542 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.550 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.551 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.552 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.553 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-08-01T04:26:52.555 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-08-01T04:26:52.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:52.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-08-01T04:26:52.560 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-08-01T04:26:52.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-08-01T04:26:52.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-08-01T04:26:52.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-08-01T04:26:52.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.564 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-08-01T04:26:52.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-08-01T04:26:52.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-08-01T04:26:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-08-01T04:26:52.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.567 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.567 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.567 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.567 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.567 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.567 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.568 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.569 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.570 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.570 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.570 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.570 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.571 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.571 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.571 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.571 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.572 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.572 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.573 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.573 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.573 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.573 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.575 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.575 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.576 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.579 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.580 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-08-01T04:26:52.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-08-01T04:26:52.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-08-01T04:26:52.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-08-01T04:26:52.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.585 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.585 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.585 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-01T04:26:52.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.588 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.589 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.589 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.589 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.589 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.589 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.589 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.590 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.591 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.592 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.594 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.595 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.596 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.596 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.596 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.596 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.597 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.597 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.597 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-08-01T04:26:52.598 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.599 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.599 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.599 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.600 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.606 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.607 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-08-01T04:26:52.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-08-01T04:26:52.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-08-01T04:26:52.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-08-01T04:26:52.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-08-01T04:26:52.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.617 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-08-01T04:26:52.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.620 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-08-01T04:26:52.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-08-01T04:26:52.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.622 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.624 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.626 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.626 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.626 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.626 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.626 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.626 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.631 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.632 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.641 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard", 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: "restful", 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_schedule" 2024-08-01T04:26:52.642 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.643 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-08-01T04:26:52.645 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.645 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.647 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.652 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.653 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-08-01T04:26:52.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-08-01T04:26:52.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.656 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.657 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-08-01T04:26:52.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.661 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.663 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-08-01T04:26:52.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-08-01T04:26:52.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-08-01T04:26:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.667 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-08-01T04:26:52.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-08-01T04:26:52.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-01T04:26:52.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-08-01T04:26:52.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-08-01T04:26:52.673 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.673 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.673 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.673 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-08-01T04:26:52.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-08-01T04:26:52.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-08-01T04:26:52.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-08-01T04:26:52.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.678 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-08-01T04:26:52.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-08-01T04:26:52.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:52.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.678 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-08-01T04:26:52.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:52.681 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-08-01T04:26:52.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.681 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-01T04:26:52.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.682 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.682 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.682 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-08-01T04:26:52.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-08-01T04:26:52.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.683 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.683 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-08-01T04:26:52.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.684 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.684 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.684 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-01T04:26:52.684 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-01T04:26:52.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.685 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-08-01T04:26:52.685 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-08-01T04:26:52.686 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-08-01T04:26:52.687 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-08-01T04:26:52.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.690 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.691 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.692 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.694 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.694 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.694 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.694 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.694 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.694 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.695 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.696 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.698 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.699 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.701 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-08-01T04:26:52.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-08-01T04:26:52.704 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.704 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-08-01T04:26:52.706 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.708 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.709 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:52.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:52.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.710 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.710 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:52.710 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:52.712 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.713 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-08-01T04:26:52.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-08-01T04:26:52.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:52.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.715 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.716 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.717 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.718 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-08-01T04:26:52.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-08-01T04:26:52.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-08-01T04:26:52.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-01T04:26:52.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.726 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-08-01T04:26:52.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-08-01T04:26:52.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.734 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-08-01T04:26:52.736 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:52.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.738 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.738 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.738 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-08-01T04:26:52.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-08-01T04:26:52.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-08-01T04:26:52.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-08-01T04:26:52.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:52.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-08-01T04:26:52.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-08-01T04:26:52.747 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-08-01T04:26:52.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-08-01T04:26:52.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-08-01T04:26:52.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.754 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-08-01T04:26:52.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-08-01T04:26:52.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.755 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-08-01T04:26:52.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:52.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-08-01T04:26:52.759 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.760 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.765 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.768 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.769 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-08-01T04:26:52.775 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.775 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-01T04:26:52.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-08-01T04:26:52.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-08-01T04:26:52.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.777 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.778 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.779 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.786 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.787 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.791 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.794 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-01T04:26:52.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.798 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.801 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.805 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.806 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.807 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-08-01T04:26:52.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.809 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.810 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-08-01T04:26:52.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-08-01T04:26:52.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.810 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.811 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-08-01T04:26:52.813 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-08-01T04:26:52.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.814 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-08-01T04:26:52.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-08-01T04:26:52.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-08-01T04:26:52.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-08-01T04:26:52.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-08-01T04:26:52.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-01T04:26:52.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.826 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-08-01T04:26:52.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-08-01T04:26:52.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-08-01T04:26:52.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-08-01T04:26:52.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.830 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-08-01T04:26:52.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-08-01T04:26:52.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-08-01T04:26:52.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-08-01T04:26:52.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-08-01T04:26:52.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.842 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-08-01T04:26:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.844 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.845 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.848 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.850 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.851 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.853 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.855 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.857 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.858 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.860 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.862 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.864 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-01T04:26:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-01T04:26:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.878 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.886 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.887 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.889 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:52.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.899 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-01T04:26:52.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-01T04:26:52.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.904 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.906 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.908 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.910 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-01T04:26:52.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-01T04:26:52.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.915 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.916 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.917 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.918 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-01T04:26:52.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.922 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.923 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.925 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-01T04:26:52.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-01T04:26:52.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.930 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.931 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-08-01T04:26:52.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.933 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-08-01T04:26:52.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.936 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:52.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.940 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-08-01T04:26:52.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-08-01T04:26:52.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.952 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.955 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.958 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:52.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-08-01T04:26:52.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.971 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-08-01T04:26:52.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:52.976 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.978 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:52.984 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-08-01T04:26:52.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.988 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-08-01T04:26:52.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-01T04:26:52.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-08-01T04:26:52.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.994 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:52.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-08-01T04:26:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:52.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:52.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.000 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.002 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.006 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-08-01T04:26:53.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-08-01T04:26:53.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-08-01T04:26:53.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.010 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-08-01T04:26:53.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-08-01T04:26:53.015 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.017 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-08-01T04:26:53.019 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:53.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.022 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.023 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.024 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.027 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.030 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.031 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-08-01T04:26:53.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.039 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-08-01T04:26:53.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.042 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.044 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.049 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.054 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.058 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.062 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-08-01T04:26:53.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.067 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.072 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.073 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.079 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.081 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.083 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-08-01T04:26:53.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-08-01T04:26:53.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.097 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.102 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.106 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.107 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.110 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.115 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.117 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.119 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.121 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.124 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.129 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.131 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.132 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.140 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.144 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.146 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.148 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.150 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.154 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.159 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.164 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:53.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.168 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-08-01T04:26:53.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.170 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.171 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-08-01T04:26:53.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.176 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.180 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-08-01T04:26:53.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-08-01T04:26:53.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.193 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.197 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-08-01T04:26:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.201 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.202 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.203 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.204 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.205 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.206 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-01T04:26:53.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.215 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.217 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.219 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.220 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.221 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-08-01T04:26:53.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-08-01T04:26:53.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.230 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-08-01T04:26:53.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.235 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.236 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.243 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.244 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.246 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.248 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.253 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-08-01T04:26:53.256 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.257 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.258 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.262 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.263 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.265 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.266 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.267 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.269 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-08-01T04:26:53.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.280 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-08-01T04:26:53.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.287 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-08-01T04:26:53.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.293 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.294 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-08-01T04:26:53.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-08-01T04:26:53.298 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.303 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.308 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-08-01T04:26:53.311 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.312 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.313 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.314 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.320 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.323 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.324 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.330 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.331 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.333 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.339 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.340 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.349 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-08-01T04:26:53.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-01T04:26:53.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.359 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-08-01T04:26:53.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-08-01T04:26:53.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.371 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.376 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.380 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.386 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.391 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.396 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.397 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.400 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.403 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-08-01T04:26:53.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.405 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.406 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-08-01T04:26:53.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-01T04:26:53.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-08-01T04:26:53.411 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-08-01T04:26:53.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.416 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.417 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.418 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-08-01T04:26:53.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-08-01T04:26:53.422 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-08-01T04:26:53.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.425 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-08-01T04:26:53.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-08-01T04:26:53.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-08-01T04:26:53.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: "services": { 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard": "https://172.21.15.3:8443/" 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2024-08-01T04:26:53.431 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-08-01T04:26:53.432 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-08-01T04:26:53.433 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.434 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-08-01T04:26:53.435 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-08-01T04:26:53.436 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-08-01T04:26:53.437 INFO:teuthology.orchestra.run.smithi003.stdout: "squid": [ 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-08-01T04:26:53.438 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 91, 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2024-08-01T04:26:53.439 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2790832943 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.440 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-08-01T04:26:53.441 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3471713361 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2749570952 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-08-01T04:26:53.443 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2398492218 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.444 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1794770455 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.445 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-08-01T04:26:53.446 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3951426736 2024-08-01T04:26:53.447 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.447 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.447 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:53.447 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-08-01T04:26:53.447 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:53.448 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:50 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2488114037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:50 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1874916929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:51 smithi163 ceph-mon[37383]: pgmap v328: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:51 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1320032553' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:50 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2488114037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:50 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1874916929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:51 smithi160 ceph-mon[37224]: pgmap v328: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-08-01T04:26:53.449 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:51 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1320032553' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-01T04:26:53.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.021+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f51780a2d00 msgr2=0x7f51780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:53.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.021+0000 7f5198a00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f51780a2d00 0x7f51780a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5180000c00 tx=0x7f5180005b50 comp rx=0 tx=0).stop 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.021+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 msgr2=0x7f519419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.021+0000 7f5198a00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 0x7f519419e580 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5184005fd0 tx=0x7f518400dde0 comp rx=0 tx=0).stop 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 shutdown_connections 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51941124a0 0x7f51941a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f51780a2d00 0x7f51780a51c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194076d70 0x7f519419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:53.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 --2- 172.21.15.3:0/1320032553 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5194076450 0x7f519419e580 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:26:53.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 >> 172.21.15.3:0/1320032553 conn(0x7f51940ff500 msgr2=0x7f51941018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:26:53.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 shutdown_connections 2024-08-01T04:26:53.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:26:51.022+0000 7f5198a00640 1 -- 172.21.15.3:0/1320032553 wait complete. 2024-08-01T04:26:53.458 DEBUG:tasks.check_counter:Getting stats from a 2024-08-01T04:26:53.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-01T04:26:53.459 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:50 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2488114037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-01T04:26:53.459 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:50 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1874916929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-01T04:26:53.459 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:51 smithi003 ceph-mon[32262]: pgmap v328: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-08-01T04:26:53.459 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:51 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1320032553' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-01T04:26:53.933 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:54.135 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:53 smithi003 ceph-mon[32262]: pgmap v329: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:54.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:53 smithi160 ceph-mon[37224]: pgmap v329: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:54.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:53 smithi163 ceph-mon[37383]: pgmap v329: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-08-01T04:26:54.677 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:54.677 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:26:54.677 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 94, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 113, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 23108, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 264876, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 31, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 5, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.088343325, 2024-08-01T04:26:54.678 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.030071312, 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.046879718, 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.001202341, 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 110, 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.046545546, 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000423141 2024-08-01T04:26:54.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 84, 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000040752, 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000485 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 4512, 2024-08-01T04:26:54.680 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 5392 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1113, 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 884, 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 305992, 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 551956, 2024-08-01T04:26:54.681 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 11, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 6, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.203615782, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.082625550, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.179448977, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.006926986, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 882, 2024-08-01T04:26:54.682 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.248473222, 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000281715 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1844, 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000727880, 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000394 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.683 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 139040 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 316, 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 421, 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1649667, 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 545603, 2024-08-01T04:26:54.684 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 12, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 5, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.134908939, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.046874622, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.138969141, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000203646, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 421, 2024-08-01T04:26:54.685 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.117294435, 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000278609 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 6, 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000913, 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000152 2024-08-01T04:26:54.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1646576, 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 463104 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2024-08-01T04:26:54.687 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 75, 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.004393950, 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000058586 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2024-08-01T04:26:54.688 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2, 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000020447, 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000010223 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2024-08-01T04:26:54.689 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 15, 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 4, 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 4, 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.003032749, 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000758187 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.690 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2024-08-01T04:26:54.691 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 11, 2024-08-01T04:26:54.691 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2024-08-01T04:26:54.691 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2024-08-01T04:26:54.691 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 11, 2024-08-01T04:26:54.691 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2024-08-01T04:26:54.691 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 22, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 0, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 22, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 0, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 22, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 1, 2024-08-01T04:26:54.692 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 3, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 31, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 14, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 11, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 4, 2024-08-01T04:26:54.693 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2024-08-01T04:26:54.694 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2024-08-01T04:26:54.694 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2024-08-01T04:26:54.694 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2024-08-01T04:26:54.694 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2024-08-01T04:26:54.694 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 0, 2024-08-01T04:26:54.694 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 1, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 2, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 7, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 325, 2024-08-01T04:26:54.695 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 13424230508, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 6, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 7, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2024-08-01T04:26:54.696 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2024-08-01T04:26:54.697 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2024-08-01T04:26:54.697 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 7, 2024-08-01T04:26:54.697 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2024-08-01T04:26:54.697 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 4, 2024-08-01T04:26:54.697 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 0, 2024-08-01T04:26:54.697 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2024-08-01T04:26:54.698 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-01T04:26:54.698 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-01T04:26:54.698 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 6, 2024-08-01T04:26:54.698 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 0, 2024-08-01T04:26:54.698 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 1, 2024-08-01T04:26:54.698 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 0 2024-08-01T04:26:54.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.699 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2024-08-01T04:26:54.699 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2024-08-01T04:26:54.699 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 3, 2024-08-01T04:26:54.699 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 2, 2024-08-01T04:26:54.699 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 6, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 0, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 0, 2024-08-01T04:26:54.700 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2024-08-01T04:26:54.701 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2024-08-01T04:26:54.701 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2024-08-01T04:26:54.701 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2024-08-01T04:26:54.701 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2024-08-01T04:26:54.701 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2024-08-01T04:26:54.702 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2024-08-01T04:26:54.702 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_path": 4, 2024-08-01T04:26:54.702 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_inode": 28, 2024-08-01T04:26:54.702 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 0, 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.703 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 13, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 13, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2024-08-01T04:26:54.704 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 2, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 2, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 2, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2024-08-01T04:26:54.705 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 4213228, 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 0, 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 12, 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.025577014, 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.002131417 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 0 2024-08-01T04:26:54.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 24, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 24, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 0, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 22, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 23, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 1, 2024-08-01T04:26:54.707 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 22, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 23, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 1, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 4, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 4, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 50584, 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 223520 2024-08-01T04:26:54.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 17, 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 0, 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 15, 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 28, 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 0, 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2024-08-01T04:26:54.709 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1, 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000351580, 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000351580 2024-08-01T04:26:54.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.711 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2, 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.002123405, 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.001061702 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.712 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.713 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.714 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.715 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.716 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.717 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2024-08-01T04:26:54.717 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.717 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2024-08-01T04:26:54.719 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1, 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000557764, 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000557764 2024-08-01T04:26:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2024-08-01T04:26:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2024-08-01T04:26:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2024-08-01T04:26:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2024-08-01T04:26:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2024-08-01T04:26:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2024-08-01T04:26:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2024-08-01T04:26:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2024-08-01T04:26:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 2, 2024-08-01T04:26:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 5, 2024-08-01T04:26:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 3, 2024-08-01T04:26:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 2, 2024-08-01T04:26:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2024-08-01T04:26:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 1, 2024-08-01T04:26:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 0, 2024-08-01T04:26:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 593, 2024-08-01T04:26:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mdthresh_evicted": 0 2024-08-01T04:26:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-08-01T04:26:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:26:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-08-01T04:26:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:26:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:26:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:26:54.735 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:26:54.736 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:26:54.737 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:26:54.738 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:26:54.738 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-08-01T04:26:54.738 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-08-01T04:26:54.738 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-08-01T04:26:54.738 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 271170, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 102, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:26:54.739 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 8512, 2024-08-01T04:26:54.740 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 187, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 105756, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 726, 2024-08-01T04:26:54.741 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2024-08-01T04:26:54.742 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 65, 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 30949, 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 33, 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 65, 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 65, 2024-08-01T04:26:54.743 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 1.065986486, 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.016399792 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 65, 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 93 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.744 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 65, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 7, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 58, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 0, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 10, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 0, 2024-08-01T04:26:54.745 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 12, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 11, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 6, 2024-08-01T04:26:54.746 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 7, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 20, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2024-08-01T04:26:54.747 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2024-08-01T04:26:54.748 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2024-08-01T04:26:54.749 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2024-08-01T04:26:54.750 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 99, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 13, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 10, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 10, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 0, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 23, 2024-08-01T04:26:54.751 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 0, 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 4 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 1, 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2024-08-01T04:26:54.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "osd-slow-ops": { 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 32 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 0, 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2024-08-01T04:26:54.753 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 0, 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed_ops": 0, 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 0, 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:54.754 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 1519, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1823454, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 1519, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-08-01T04:26:54.755 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 1519, 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1823454, 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.756 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 65, 2024-08-01T04:26:54.757 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 30949, 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 65, 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 30949, 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-08-01T04:26:54.759 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 65, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 65, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 65, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 65, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.760 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.761 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.761 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.761 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2024-08-01T04:26:54.761 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-08-01T04:26:54.762 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-08-01T04:26:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-08-01T04:26:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-08-01T04:26:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.763 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x55f49c676960": { 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:54.764 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 13, 2024-08-01T04:26:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 18924, 2024-08-01T04:26:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 13, 2024-08-01T04:26:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-08-01T04:26:54.765 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 12, 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 18924, 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:54.766 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.767 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:54.767 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:55.788 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-08-01T04:26:55.788 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2024-08-01T04:26:55.788 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-08-01T04:26:55.788 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-08-01T04:26:55.789 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-08-01T04:26:55.789 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-08-01T04:26:55.789 DEBUG:tasks.check_counter:Getting stats from d 2024-08-01T04:26:55.789 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-01T04:26:56.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:55 smithi003 ceph-mon[32262]: pgmap v330: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:26:56.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:55 smithi160 ceph-mon[37224]: pgmap v330: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:26:56.284 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:56.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:55 smithi163 ceph-mon[37383]: pgmap v330: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:26:56.984 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 235, 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 222, 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1586400, 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 135536, 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-08-01T04:26:56.985 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.098023199, 2024-08-01T04:26:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.024412878, 2024-08-01T04:26:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.115005209, 2024-08-01T04:26:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:26:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 222, 2024-08-01T04:26:56.986 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.044861236, 2024-08-01T04:26:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000202077 2024-08-01T04:26:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:56.987 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1586400, 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 135536 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-08-01T04:26:56.988 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 169, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 186368, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.033389588, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.024295432, 2024-08-01T04:26:56.989 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.004536096, 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 166, 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.047158412, 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000284086 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.990 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 186368 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.991 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001378815, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-08-01T04:26:56.992 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001098408, 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.993 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-01T04:26:56.994 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-08-01T04:26:56.995 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:26:56.996 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:26:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:26:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:26:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:26:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:26:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:26:56.997 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:26:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:26:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-08-01T04:26:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:26:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-08-01T04:26:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:26:56.998 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-08-01T04:26:56.999 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:26:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:26:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:26:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-08-01T04:26:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-08-01T04:26:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 150008, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-08-01T04:26:57.001 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:26:57.002 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-08-01T04:26:57.003 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-08-01T04:26:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-08-01T04:26:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-08-01T04:26:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-08-01T04:26:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-08-01T04:26:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:57.004 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:26:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-08-01T04:26:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 233, 2024-08-01T04:26:57.005 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1550830, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 233, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 233, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1550830, 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:57.006 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:57.007 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:57.007 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:57.807 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-01T04:26:57.808 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-01T04:26:57.808 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-01T04:26:57.808 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-01T04:26:57.808 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-01T04:26:57.808 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-08-01T04:26:57.808 DEBUG:tasks.check_counter:Getting stats from g 2024-08-01T04:26:57.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-01T04:26:58.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:57 smithi003 ceph-mon[32262]: pgmap v331: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:58.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:57 smithi160 ceph-mon[37224]: pgmap v331: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:58.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:57 smithi163 ceph-mon[37383]: pgmap v331: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-01T04:26:58.298 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:26:59.010 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 4, 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 3024, 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 448, 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-08-01T04:26:59.011 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.007511485, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000113716, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.004096602, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.007783711, 2024-08-01T04:26:59.012 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.002594570 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-01T04:26:59.013 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 228, 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 217, 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1582560, 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 133168, 2024-08-01T04:26:59.014 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.088238384, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.035741576, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.077369722, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 216, 2024-08-01T04:26:59.015 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.044089439, 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000204117 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1582560, 2024-08-01T04:26:59.016 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 133168 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 165, 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 185664, 2024-08-01T04:26:59.017 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-08-01T04:26:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-08-01T04:26:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.032909907, 2024-08-01T04:26:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.022736071, 2024-08-01T04:26:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.004684960, 2024-08-01T04:26:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:26:59.018 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 162, 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.039585275, 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000244353 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:59.019 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 185664 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-08-01T04:26:59.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:26:59.021 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:26:59.022 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:26:59.023 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:26:59.024 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148977, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 30, 2024-08-01T04:26:59.025 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-08-01T04:26:59.026 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-08-01T04:26:59.027 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-08-01T04:26:59.028 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 230, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1550440, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 230, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 230, 2024-08-01T04:26:59.029 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1550440, 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:26:59.030 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:26:59.853 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-08-01T04:26:59.854 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-08-01T04:26:59.854 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-08-01T04:26:59.854 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-08-01T04:26:59.854 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-08-01T04:26:59.854 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-08-01T04:26:59.854 DEBUG:tasks.check_counter:Getting stats from j 2024-08-01T04:26:59.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-01T04:27:00.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:26:59 smithi003 ceph-mon[32262]: pgmap v332: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:27:00.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:26:59 smithi160 ceph-mon[37224]: pgmap v332: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:27:00.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:26:59 smithi163 ceph-mon[37383]: pgmap v332: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-01T04:27:00.339 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:01.074 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-08-01T04:27:01.074 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:01.074 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 233, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 222, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1583920, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 136368, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.109507125, 2024-08-01T04:27:01.075 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.025274870, 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.119722064, 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 221, 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.053922131, 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000243991 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1583920, 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 136368 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:01.077 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 6, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 172, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 3344, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 186992, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.035741215, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.022667594, 2024-08-01T04:27:01.078 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.008691865, 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 170, 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.059827962, 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000351929 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:01.079 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 186992 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-08-01T04:27:01.080 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.000977112, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000805743, 2024-08-01T04:27:01.081 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:01.082 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:01.082 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:27:01.082 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:27:01.082 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:27:01.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-08-01T04:27:01.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:01.084 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:01.085 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:01.086 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:01.087 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:01.088 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 149745, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-08-01T04:27:01.089 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:01.090 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:01.091 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 235, 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1551090, 2024-08-01T04:27:01.092 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 235, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 235, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1551090, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-08-01T04:27:01.093 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-08-01T04:27:01.094 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-08-01T04:27:01.094 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:27:01.094 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-08-01T04:27:01.094 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-08-01T04:27:02.023 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-01T04:27:02.024 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-01T04:27:02.024 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-08-01T04:27:02.024 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-08-01T04:27:02.024 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-08-01T04:27:02.024 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-08-01T04:27:02.024 DEBUG:tasks.check_counter:Getting stats from b 2024-08-01T04:27:02.024 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-01T04:27:02.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:01 smithi160 ceph-mon[37224]: pgmap v333: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:27:02.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:01 smithi163 ceph-mon[37383]: pgmap v333: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:27:02.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:01 smithi003 ceph-mon[32262]: pgmap v333: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-01T04:27:02.361 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:27:03.135 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-08-01T04:27:03.135 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2872, 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 3259, 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 2468172, 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 21785625, 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 29, 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-08-01T04:27:03.136 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.300435969, 2024-08-01T04:27:03.137 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.123925299, 2024-08-01T04:27:03.137 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.133812239, 2024-08-01T04:27:03.137 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.050614602, 2024-08-01T04:27:03.137 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:03.137 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3261, 2024-08-01T04:27:03.137 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.242546689, 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000074378 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1711, 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000425714, 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000248 2024-08-01T04:27:03.138 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1651248, 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 477600 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 18011, 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 21396, 2024-08-01T04:27:03.139 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 6021200, 2024-08-01T04:27:03.140 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 17752684, 2024-08-01T04:27:03.140 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 86, 2024-08-01T04:27:03.140 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 8, 2024-08-01T04:27:03.140 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 1.061240019, 2024-08-01T04:27:03.140 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.426742426, 2024-08-01T04:27:03.140 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.762577580, 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.101619740, 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 21393, 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2.706054028, 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000126492 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.141 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 14006, 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.004187609, 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000298 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 146032, 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 253024 2024-08-01T04:27:03.142 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.143 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:03.143 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4004, 2024-08-01T04:27:03.143 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 3752, 2024-08-01T04:27:03.143 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1351465, 2024-08-01T04:27:03.145 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 20481816, 2024-08-01T04:27:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 18, 2024-08-01T04:27:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 8, 2024-08-01T04:27:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.403191552, 2024-08-01T04:27:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.155585743, 2024-08-01T04:27:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.139578861, 2024-08-01T04:27:03.146 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.101369608, 2024-08-01T04:27:03.147 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:03.147 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3748, 2024-08-01T04:27:03.147 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.470596067, 2024-08-01T04:27:03.147 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000125559 2024-08-01T04:27:03.147 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2848, 2024-08-01T04:27:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000585344, 2024-08-01T04:27:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000205 2024-08-01T04:27:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.148 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 323584, 2024-08-01T04:27:03.149 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 98944 2024-08-01T04:27:03.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.149 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-08-01T04:27:03.149 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-08-01T04:27:03.149 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-08-01T04:27:03.149 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6352, 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 5.603242666, 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000882122 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.150 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 835, 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.048953083, 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000058626 2024-08-01T04:27:03.151 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 2304, 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 2300, 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2300, 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 26.359678939, 2024-08-01T04:27:03.152 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.011460729 2024-08-01T04:27:03.153 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.153 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-08-01T04:27:03.153 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 4, 2024-08-01T04:27:03.153 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-08-01T04:27:03.153 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-08-01T04:27:03.153 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 29, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 2, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 1150, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 97, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 1026, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 27, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 1012, 2024-08-01T04:27:03.154 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 12, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 147, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 147, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 8, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 2910, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 2792, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-08-01T04:27:03.155 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 48, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 4, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 8, 2024-08-01T04:27:03.156 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-08-01T04:27:03.157 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-08-01T04:27:03.157 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-08-01T04:27:03.157 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 2709, 2024-08-01T04:27:03.157 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-08-01T04:27:03.157 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 325, 2024-08-01T04:27:03.157 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 13424230508, 2024-08-01T04:27:03.158 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 6, 2024-08-01T04:27:03.158 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 2709, 2024-08-01T04:27:03.158 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-08-01T04:27:03.158 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-08-01T04:27:03.158 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-08-01T04:27:03.158 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 649, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 10, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 639, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 2060, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 8, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 1049, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 3678, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 6, 2024-08-01T04:27:03.159 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 12831, 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 10993, 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 184, 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 916 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-08-01T04:27:03.160 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 2, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 14, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-08-01T04:27:03.161 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 530, 2024-08-01T04:27:03.162 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 469, 2024-08-01T04:27:03.162 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 61, 2024-08-01T04:27:03.162 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-08-01T04:27:03.162 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-08-01T04:27:03.162 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 4, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 846, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 10, 2024-08-01T04:27:03.163 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 4, 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 2, 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-08-01T04:27:03.164 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 14241, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 14241, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 21, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-08-01T04:27:03.165 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 21, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 3, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 56696702, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-08-01T04:27:03.166 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2213, 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 376.080177123, 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.169941336 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 933, 2024-08-01T04:27:03.167 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 1416, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 483, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 115, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 133, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 18, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 1150, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 1689, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 539, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 147, 2024-08-01T04:27:03.168 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 1056, 2024-08-01T04:27:03.169 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 909, 2024-08-01T04:27:03.169 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 73968, 2024-08-01T04:27:03.169 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 231712 2024-08-01T04:27:03.169 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.169 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-08-01T04:27:03.169 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 2682, 2024-08-01T04:27:03.170 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 1, 2024-08-01T04:27:03.170 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 2304, 2024-08-01T04:27:03.170 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 36, 2024-08-01T04:27:03.170 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 2, 2024-08-01T04:27:03.170 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-08-01T04:27:03.170 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 765, 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2.577443821, 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.003369207 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 176, 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.095020963, 2024-08-01T04:27:03.171 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000539891 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.172 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 61, 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.047198837, 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000773751 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 69, 2024-08-01T04:27:03.173 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.234374081, 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.003396725 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.174 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000306376, 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000306376 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-08-01T04:27:03.175 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.176 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.177 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.177 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-08-01T04:27:03.177 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.177 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.177 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.178 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.178 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-08-01T04:27:03.179 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 11, 2024-08-01T04:27:03.179 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.002393242, 2024-08-01T04:27:03.179 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000217567 2024-08-01T04:27:03.179 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.179 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-08-01T04:27:03.180 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 105, 2024-08-01T04:27:03.180 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 11.133139552, 2024-08-01T04:27:03.180 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.106029900 2024-08-01T04:27:03.180 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.180 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 11, 2024-08-01T04:27:03.181 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.193711407, 2024-08-01T04:27:03.182 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.017610127 2024-08-01T04:27:03.182 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.182 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-08-01T04:27:03.182 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 298, 2024-08-01T04:27:03.182 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.283236968, 2024-08-01T04:27:03.182 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000950459 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 156, 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.899988839, 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005769159 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 113, 2024-08-01T04:27:03.183 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 10.054917662, 2024-08-01T04:27:03.184 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.088981572 2024-08-01T04:27:03.184 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.184 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-08-01T04:27:03.184 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.184 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-08-01T04:27:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.185 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 16, 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.149780831, 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.009361301 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-08-01T04:27:03.186 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5, 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.037360498, 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.007472099 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.187 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 7, 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.038193449, 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005456207 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.188 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4, 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001342369, 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000335592 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.189 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 8, 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.019443961, 2024-08-01T04:27:03.190 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002430495 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 13, 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.008500594, 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000653891 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-08-01T04:27:03.191 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 481, 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.583325489, 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001212734 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.192 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 5, 2024-08-01T04:27:03.193 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 3, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 38, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 19, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 623, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.194 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:03.195 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:03.196 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:03.197 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:03.198 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 326021, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 510, 2024-08-01T04:27:03.199 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8512, 2024-08-01T04:27:03.200 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 187, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 3246901, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 36216, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:03.201 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 6315, 2024-08-01T04:27:03.202 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 52934464, 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 32, 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 6315, 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6315, 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 486.174096568, 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.076987188 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.203 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 6315, 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 6654 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 6315, 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2746, 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 3569, 2024-08-01T04:27:03.204 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 0, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 11, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 27, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 2230, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 51, 2024-08-01T04:27:03.205 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-08-01T04:27:03.206 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-08-01T04:27:03.206 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-08-01T04:27:03.206 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 1, 2024-08-01T04:27:03.206 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 994, 2024-08-01T04:27:03.206 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 2719, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 22, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-08-01T04:27:03.207 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-08-01T04:27:03.208 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-08-01T04:27:03.208 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-08-01T04:27:03.208 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-08-01T04:27:03.208 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-08-01T04:27:03.208 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-08-01T04:27:03.208 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 6, 2024-08-01T04:27:03.209 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-08-01T04:27:03.209 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-08-01T04:27:03.209 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-08-01T04:27:03.209 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-08-01T04:27:03.209 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-08-01T04:27:03.209 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-08-01T04:27:03.210 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-08-01T04:27:03.210 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-08-01T04:27:03.210 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-08-01T04:27:03.210 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-08-01T04:27:03.210 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-08-01T04:27:03.211 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-08-01T04:27:03.211 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-08-01T04:27:03.211 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-08-01T04:27:03.211 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-08-01T04:27:03.211 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-08-01T04:27:03.211 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 99, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 13, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-08-01T04:27:03.212 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 408, 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 185 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-08-01T04:27:03.213 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-08-01T04:27:03.214 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-08-01T04:27:03.214 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.214 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-08-01T04:27:03.214 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 852 2024-08-01T04:27:03.214 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.214 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 10, 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 10, 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 469, 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 469, 2024-08-01T04:27:03.215 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-08-01T04:27:03.216 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.216 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:03.216 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:03.216 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:03.216 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:03.217 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 24883, 2024-08-01T04:27:03.217 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 7780003, 2024-08-01T04:27:03.217 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:03.217 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 24883, 2024-08-01T04:27:03.217 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:03.217 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:03.218 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 24883, 2024-08-01T04:27:03.218 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 7780003, 2024-08-01T04:27:03.218 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:03.218 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.218 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.218 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.219 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.219 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.219 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-08-01T04:27:03.219 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:03.219 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:03.219 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 6315, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 52981833, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6315, 2024-08-01T04:27:03.220 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 52981833, 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-08-01T04:27:03.221 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 6315, 2024-08-01T04:27:03.222 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 6315, 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6315, 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 6315, 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.223 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 469, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 47369, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:03.224 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 469, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 17, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 47369, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.225 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x559a83db6960": { 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 60, 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 14241, 2024-08-01T04:27:03.226 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 52502398, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 14241, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2213, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 52502338, 2024-08-01T04:27:03.227 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:03.228 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:03.228 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:03.228 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:03.228 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.228 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:03.228 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-08-01T04:27:04.140 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:03 smithi160 ceph-mon[37224]: pgmap v334: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:27:04.143 INFO:tasks.check_counter:Daemon mds.b mds.exported=4 2024-08-01T04:27:04.143 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-08-01T04:27:04.143 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-01T04:27:04.143 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-08-01T04:27:04.143 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-08-01T04:27:04.143 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=2 2024-08-01T04:27:04.143 DEBUG:tasks.check_counter:Getting stats from e 2024-08-01T04:27:04.144 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-01T04:27:04.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:03 smithi163 ceph-mon[37383]: pgmap v334: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:27:04.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:03 smithi003 ceph-mon[32262]: pgmap v334: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:27:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:27:04.902 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-08-01T04:27:04.902 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:04.902 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 749, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 751, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 248622, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 238246, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.068477796, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.027298063, 2024-08-01T04:27:04.903 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.016173926, 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.017930435, 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 749, 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.031494178, 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000042048 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.904 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.905 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 897, 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 879, 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1786500, 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 342523, 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:04.906 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.107221271, 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.031005085, 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.059424306, 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.025676594, 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 879, 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.042720829, 2024-08-01T04:27:04.907 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000048601 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1629776, 2024-08-01T04:27:04.908 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 139952 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 736, 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 903, 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 246944, 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 687204, 2024-08-01T04:27:04.909 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.098604056, 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.033391744, 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.024801947, 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.030456720, 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:04.910 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 900, 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.066449695, 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000073832 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.911 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 453792 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-08-01T04:27:04.912 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3596, 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.104660974, 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000029104 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.913 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.914 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.915 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-08-01T04:27:04.916 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 18, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 0, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 18, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 18, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2024-08-01T04:27:04.917 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 5, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-08-01T04:27:04.918 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-08-01T04:27:04.919 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-08-01T04:27:04.920 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2024-08-01T04:27:04.921 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2024-08-01T04:27:04.922 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-08-01T04:27:04.923 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-08-01T04:27:04.924 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-08-01T04:27:04.925 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-08-01T04:27:04.926 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 22, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 2, 2024-08-01T04:27:04.927 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 4246241, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 4245667, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.928 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 22 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 18, 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 18, 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 0, 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 14, 2024-08-01T04:27:04.929 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 14, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 0, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 18, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 18, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 0, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2024-08-01T04:27:04.930 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 51180, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 215328 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2024-08-01T04:27:04.931 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.932 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.933 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.934 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.935 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.936 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-08-01T04:27:04.938 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.939 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.939 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-08-01T04:27:04.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.942 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.943 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.944 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.945 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.946 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-08-01T04:27:04.947 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.948 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.949 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.950 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.951 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 5, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 3, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 4, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2024-08-01T04:27:04.952 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 3696, 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:04.953 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:04.954 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:04.955 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:04.956 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:04.957 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 189892, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 65, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-08-01T04:27:04.958 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8512, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 187, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:04.959 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 79452, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 898, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-08-01T04:27:04.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 2120, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 2120, 2024-08-01T04:27:04.961 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2120, 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 44.946361975, 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.021201114 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2120, 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2121 2024-08-01T04:27:04.962 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 2120, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2120, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 1369, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2024-08-01T04:27:04.963 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 749, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-08-01T04:27:04.964 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-08-01T04:27:04.965 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-08-01T04:27:04.966 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-08-01T04:27:04.967 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 99, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-08-01T04:27:04.968 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 13, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 5, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 5, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2024-08-01T04:27:04.969 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-08-01T04:27:04.970 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 0, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-08-01T04:27:04.971 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 2378, 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 2051931, 2024-08-01T04:27:04.972 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 2378, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2378, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2051931, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.973 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.974 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.974 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.974 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-08-01T04:27:04.974 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:04.974 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:04.974 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2120, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 51942, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2120, 2024-08-01T04:27:04.975 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 51942, 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-08-01T04:27:04.976 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:04.977 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2120, 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 2120, 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2120, 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2120, 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.978 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:04.979 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.980 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x563a27b52b40": { 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:04.981 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-08-01T04:27:04.982 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:04.983 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-08-01T04:27:05.669 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-01T04:27:05.669 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-01T04:27:05.669 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-08-01T04:27:05.669 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-08-01T04:27:05.670 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-08-01T04:27:05.670 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-08-01T04:27:05.670 DEBUG:tasks.check_counter:Getting stats from h 2024-08-01T04:27:05.670 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-01T04:27:05.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:05 smithi160 ceph-mon[37224]: pgmap v335: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:27:05.927 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:27:06.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:05 smithi163 ceph-mon[37383]: pgmap v335: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:27:06.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:05 smithi003 ceph-mon[32262]: pgmap v335: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-08-01T04:27:06.465 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-08-01T04:27:06.465 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:06.465 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1548, 2024-08-01T04:27:06.465 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1242, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1991475, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 665422, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 10, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.144330770, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.060790193, 2024-08-01T04:27:06.466 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.100421160, 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.004370923, 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1244, 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.079118158, 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000063599 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:06.467 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2060, 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000410632, 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000199 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1647936, 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 166656 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:06.468 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 64, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 103, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 14384, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 502550, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 33, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 7, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.016770899, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.003222164, 2024-08-01T04:27:06.469 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.011511498, 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000626205, 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 99, 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.017019961, 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000171918 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:06.470 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 48, 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000007353, 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000153 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3888, 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 215856 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:06.471 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 70, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 232, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 19368, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 374185, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.019224339, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.009146621, 2024-08-01T04:27:06.472 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.007759361, 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000753464, 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 228, 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.048929831, 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000214604 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:06.473 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 55, 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000037771, 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000686 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 322112 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.474 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 105, 2024-08-01T04:27:06.475 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003827990, 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000036457 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-08-01T04:27:06.476 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000009047, 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000004523 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 18, 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 10, 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-08-01T04:27:06.477 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 10, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003522998, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000352299 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 8, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-08-01T04:27:06.478 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 11, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 28, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 0, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 28, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-08-01T04:27:06.479 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 28, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 1, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 1, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 2, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 5, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 39, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 22, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 6, 2024-08-01T04:27:06.480 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 8, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-08-01T04:27:06.481 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 3, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 6, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 21, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 325, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 13424230508, 2024-08-01T04:27:06.482 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 6, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 21, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 21, 2024-08-01T04:27:06.483 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 10, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 16, 2024-08-01T04:27:06.484 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 1, 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 3, 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 2, 2024-08-01T04:27:06.485 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 8, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-08-01T04:27:06.486 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 4, 2024-08-01T04:27:06.487 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 36, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.488 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 22, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 22, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-08-01T04:27:06.489 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 2, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 2, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 2, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2024-08-01T04:27:06.490 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 4246241, 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 15, 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.027072755, 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001804850 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-08-01T04:27:06.491 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.492 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-08-01T04:27:06.492 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 28, 2024-08-01T04:27:06.492 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 28, 2024-08-01T04:27:06.492 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 0, 2024-08-01T04:27:06.492 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 24, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 25, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 1, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 28, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 29, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 1, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 2, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 10, 2024-08-01T04:27:06.493 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 8, 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 53576, 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 223520 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 23, 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 18, 2024-08-01T04:27:06.494 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 30, 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 1, 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-08-01T04:27:06.495 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4, 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.001783887, 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000445971 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2024-08-01T04:27:06.497 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000907837, 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000453918 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.498 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000106924, 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000106924 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.499 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.500 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.501 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.502 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.502 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.503 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.504 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000724350, 2024-08-01T04:27:06.505 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000241450 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-08-01T04:27:06.506 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.507 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.508 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.509 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.510 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.511 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.512 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-08-01T04:27:06.513 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.514 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 5, 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 3, 2024-08-01T04:27:06.515 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 5, 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 2, 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 608, 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-08-01T04:27:06.516 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:06.517 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:06.518 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:06.519 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:06.520 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 263974, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 102, 2024-08-01T04:27:06.521 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:06.522 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 8512, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 187, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 123476, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 1193, 2024-08-01T04:27:06.523 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-08-01T04:27:06.524 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 82, 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 64410, 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 33, 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 82, 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 82, 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.549992242, 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006707222 2024-08-01T04:27:06.525 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.526 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-08-01T04:27:06.526 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-08-01T04:27:06.526 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 82, 2024-08-01T04:27:06.526 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 110 2024-08-01T04:27:06.526 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 82, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 21, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 61, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 0, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 10, 2024-08-01T04:27:06.527 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 0, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 15, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 11, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 6, 2024-08-01T04:27:06.528 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 21, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 20, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-08-01T04:27:06.529 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-08-01T04:27:06.530 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-08-01T04:27:06.531 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-08-01T04:27:06.532 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 99, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 13, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 9, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 9, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 23, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-08-01T04:27:06.533 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 4 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-08-01T04:27:06.534 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 40 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-08-01T04:27:06.535 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 0, 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:06.536 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 1678, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1855112, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 1678, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1678, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1855112, 2024-08-01T04:27:06.537 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:06.538 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 82, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 64410, 2024-08-01T04:27:06.539 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 82, 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 64410, 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:06.541 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 82, 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 82, 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 82, 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 82, 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.542 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-08-01T04:27:06.543 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.544 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x561d2c55e960": { 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:06.545 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 22, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 51937, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 22, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 15, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 51937, 2024-08-01T04:27:06.546 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:06.547 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:06.547 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:06.547 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:06.547 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.547 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:06.547 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-08-01T04:27:07.189 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-08-01T04:27:07.189 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-08-01T04:27:07.189 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-08-01T04:27:07.189 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-01T04:27:07.189 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-01T04:27:07.190 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-08-01T04:27:07.190 DEBUG:tasks.check_counter:Getting stats from k 2024-08-01T04:27:07.190 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-01T04:27:07.638 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.b/config 2024-08-01T04:27:08.050 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:07 smithi160 ceph-mon[37224]: pgmap v336: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:27:08.050 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:07 smithi160 ceph-mon[37224]: osdmap e100: 12 total, 12 up, 12 in 2024-08-01T04:27:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:07 smithi163 ceph-mon[37383]: pgmap v336: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:27:08.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:07 smithi163 ceph-mon[37383]: osdmap e100: 12 total, 12 up, 12 in 2024-08-01T04:27:08.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:07 smithi003 ceph-mon[32262]: pgmap v336: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-08-01T04:27:08.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:07 smithi003 ceph-mon[32262]: osdmap e100: 12 total, 12 up, 12 in 2024-08-01T04:27:08.350 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-08-01T04:27:08.350 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:08.350 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 236, 2024-08-01T04:27:08.350 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 225, 2024-08-01T04:27:08.350 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1584736, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 138304, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.046223331, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.014748125, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.045670993, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:08.351 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 224, 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.012595689, 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000056230 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:08.352 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1584736, 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 138304 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2, 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 173, 2024-08-01T04:27:08.353 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 320, 2024-08-01T04:27:08.354 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 187072, 2024-08-01T04:27:08.354 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 5, 2024-08-01T04:27:08.354 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:08.354 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.011249223, 2024-08-01T04:27:08.354 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.007860889, 2024-08-01T04:27:08.354 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001226387, 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 170, 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.022906801, 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000134745 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:08.355 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 187072 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2024-08-01T04:27:08.356 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 3024, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.002211637, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000033523, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.002149245, 2024-08-01T04:27:08.357 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003975406, 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001325135 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:08.358 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-08-01T04:27:08.359 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:08.360 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:08.361 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:08.362 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:08.363 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148714, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 30, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:08.364 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-08-01T04:27:08.365 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-08-01T04:27:08.366 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:08.366 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:08.366 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:08.366 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-08-01T04:27:08.366 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-08-01T04:27:08.367 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 238, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1551480, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 238, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-08-01T04:27:08.368 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 238, 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1551480, 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-08-01T04:27:08.369 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-08-01T04:27:09.256 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-01T04:27:09.256 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-01T04:27:09.256 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-08-01T04:27:09.256 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-08-01T04:27:09.256 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-08-01T04:27:09.256 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-08-01T04:27:09.256 DEBUG:tasks.check_counter:Getting stats from c 2024-08-01T04:27:09.257 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-01T04:27:09.836 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:27:10.229 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:09 smithi163 ceph-mon[37383]: pgmap v338: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:27:10.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:09 smithi003 ceph-mon[32262]: pgmap v338: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:27:10.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:09 smithi160 ceph-mon[37224]: pgmap v338: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:27:11.232 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-08-01T04:27:11.232 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1022, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1006, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 1884656, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 382950, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.134797591, 2024-08-01T04:27:11.233 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.042495262, 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.067729099, 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.030417415, 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1005, 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.051296687, 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000051041 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.234 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 1627616, 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 139680 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.235 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 4, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 171, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 876, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 456106, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 6, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 4, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.010625434, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.006976968, 2024-08-01T04:27:11.236 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.001715083, 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000071331, 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 168, 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.025353797, 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000150915 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:11.237 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.238 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.238 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.238 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.238 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 455408 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1449, 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1449, 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 390303, 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 459965, 2024-08-01T04:27:11.239 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.104513678, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.032060774, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.029862211, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.036061609, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1448, 2024-08-01T04:27:11.240 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.061865266, 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000042724 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.241 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 448 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-08-01T04:27:11.242 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 3697, 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.098980351, 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000026773 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-08-01T04:27:11.243 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-08-01T04:27:11.244 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-08-01T04:27:11.245 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 12, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 0, 2024-08-01T04:27:11.246 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 12, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 12, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 3, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2024-08-01T04:27:11.247 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-08-01T04:27:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 0, 2024-08-01T04:27:11.249 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2024-08-01T04:27:11.250 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-01T04:27:11.251 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-08-01T04:27:11.252 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-08-01T04:27:11.253 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-08-01T04:27:11.254 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-08-01T04:27:11.255 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 13, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-08-01T04:27:11.256 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 2, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 2, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 4213228, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 4212654, 2024-08-01T04:27:11.257 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 13 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-08-01T04:27:11.258 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 14, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 14, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 0, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 12, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 12, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 0, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 12, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 12, 2024-08-01T04:27:11.259 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 0, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 48836, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 215328 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2024-08-01T04:27:11.260 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.261 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-08-01T04:27:11.262 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.263 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.264 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.265 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-08-01T04:27:11.266 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.266 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.266 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.266 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.266 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.267 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.268 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-08-01T04:27:11.269 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.269 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.269 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.270 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.271 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.272 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.273 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.274 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.275 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.276 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.276 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-08-01T04:27:11.276 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.276 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.276 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.276 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.277 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.278 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.279 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-08-01T04:27:11.280 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.281 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 2, 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 5, 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 3, 2024-08-01T04:27:11.282 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 2, 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 5120, 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-08-01T04:27:11.283 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:11.284 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:11.285 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:11.286 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:11.286 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:11.286 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:11.286 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:11.286 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:11.287 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:11.288 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:11.288 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-08-01T04:27:11.288 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:11.288 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:11.288 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:11.289 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:11.289 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 181624, 2024-08-01T04:27:11.289 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 56, 2024-08-01T04:27:11.289 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:11.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:11.291 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 8632, 2024-08-01T04:27:11.291 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 190, 2024-08-01T04:27:11.291 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:11.291 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:11.291 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:11.292 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-08-01T04:27:11.292 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 63084, 2024-08-01T04:27:11.292 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 492, 2024-08-01T04:27:11.292 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:11.292 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 2212, 2024-08-01T04:27:11.293 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 21, 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 3, 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 2212, 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2212, 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 29.250359207, 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.013223489 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.294 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2212, 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 2213 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 2212, 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 2212, 2024-08-01T04:27:11.295 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 1431, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 779, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2024-08-01T04:27:11.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-08-01T04:27:11.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 1, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-08-01T04:27:11.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-08-01T04:27:11.299 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-08-01T04:27:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-08-01T04:27:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-08-01T04:27:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-08-01T04:27:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-08-01T04:27:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-08-01T04:27:11.300 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-08-01T04:27:11.301 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-08-01T04:27:11.301 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-08-01T04:27:11.301 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-08-01T04:27:11.301 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-08-01T04:27:11.301 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-08-01T04:27:11.301 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-08-01T04:27:11.302 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 100, 2024-08-01T04:27:11.302 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-08-01T04:27:11.302 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 14, 2024-08-01T04:27:11.302 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 4, 2024-08-01T04:27:11.302 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 4, 2024-08-01T04:27:11.303 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-08-01T04:27:11.303 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-08-01T04:27:11.303 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2024-08-01T04:27:11.303 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2024-08-01T04:27:11.303 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2024-08-01T04:27:11.303 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-08-01T04:27:11.304 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 0, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2024-08-01T04:27:11.305 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-08-01T04:27:11.306 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.306 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:11.306 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:11.306 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-08-01T04:27:11.306 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:11.306 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 2471, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 2037279, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 2471, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2471, 2024-08-01T04:27:11.307 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 2037279, 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-08-01T04:27:11.308 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 2212, 2024-08-01T04:27:11.309 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 18929, 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2212, 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 18929, 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.310 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:11.311 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 2212, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 2212, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2212, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 2212, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.312 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:11.313 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-08-01T04:27:11.314 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-08-01T04:27:11.315 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:11.315 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.315 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.315 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x561c562f2b40": { 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:11.316 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:11.317 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:11.317 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:11.317 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:11.317 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:11.318 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-08-01T04:27:11.318 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-08-01T04:27:11.318 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:11.318 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:11.318 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:11.319 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:11.319 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.319 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:11.319 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-08-01T04:27:12.270 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:11 smithi163 ceph-mon[37383]: pgmap v339: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:27:12.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:11 smithi003 ceph-mon[32262]: pgmap v339: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:27:12.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:11 smithi160 ceph-mon[37224]: pgmap v339: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-08-01T04:27:12.803 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-01T04:27:12.803 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-01T04:27:12.803 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-08-01T04:27:12.803 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-01T04:27:12.803 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-01T04:27:12.804 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-08-01T04:27:12.804 DEBUG:tasks.check_counter:Getting stats from f 2024-08-01T04:27:12.804 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-01T04:27:13.066 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:27:14.003 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:13 smithi163 ceph-mon[37383]: pgmap v340: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:27:14.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:13 smithi003 ceph-mon[32262]: pgmap v340: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:27:14.335 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-08-01T04:27:14.335 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 0, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 0, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 0, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 0, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.000936450, 2024-08-01T04:27:14.336 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.000000000, 2024-08-01T04:27:14.337 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.000103211, 2024-08-01T04:27:14.337 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:14.337 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:14.337 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:14.338 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:14.338 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:14.338 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.338 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:14.338 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:14.338 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:14.339 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:14.339 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.339 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-01T04:27:14.339 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-08-01T04:27:14.339 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.340 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:14.340 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 6, 2024-08-01T04:27:14.340 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 173, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 3344, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 187168, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.013297622, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.007613217, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.003712912, 2024-08-01T04:27:14.341 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 171, 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.024032967, 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000140543 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:14.342 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 187168 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:14.343 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 234, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 223, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 1584192, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 137024, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 2, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.042141213, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.014709151, 2024-08-01T04:27:14.344 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.041590511, 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 222, 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.020405312, 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000091915 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:14.345 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 1584192, 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 137024 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-08-01T04:27:14.346 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 0, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:14.347 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:14.348 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:14.349 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:14.350 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:14.351 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:14.351 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-08-01T04:27:14.351 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:14.351 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:14.351 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:14.351 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 148995, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 31, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:14.352 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:14.353 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 928, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 2, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:14.354 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 236, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 1551220, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 236, 2024-08-01T04:27:14.355 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 236, 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 1551220, 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:14.356 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:14.357 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:14.357 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-08-01T04:27:14.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:13 smithi160 ceph-mon[37224]: pgmap v340: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:27:15.323 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-08-01T04:27:15.323 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-08-01T04:27:15.323 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-08-01T04:27:15.323 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-08-01T04:27:15.323 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-08-01T04:27:15.324 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-08-01T04:27:15.324 DEBUG:tasks.check_counter:Getting stats from i 2024-08-01T04:27:15.324 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-01T04:27:15.585 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:27:16.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:15 smithi163 ceph-mon[37383]: pgmap v341: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:27:16.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:15 smithi003 ceph-mon[32262]: pgmap v341: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:27:16.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:15 smithi160 ceph-mon[37224]: pgmap v341: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-08-01T04:27:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:17 smithi163 ceph-mon[37383]: pgmap v342: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:27:18.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:17 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:27:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:17 smithi003 ceph-mon[32262]: pgmap v342: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:27:18.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:17 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:27:18.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:17 smithi160 ceph-mon[37224]: pgmap v342: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-01T04:27:18.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:17 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-01T04:27:18.666 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-08-01T04:27:18.666 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:18.666 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 1558, 2024-08-01T04:27:18.666 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 1560, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 23619320, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 495454, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 7, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.165963483, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.049799034, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.053460408, 2024-08-01T04:27:18.667 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.049157797, 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 1558, 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.069789045, 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000044793 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.668 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:18.669 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 358, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 524, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 21068767, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 571266, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 8, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 6, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.055054659, 2024-08-01T04:27:18.670 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.014391005, 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.026699781, 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.009918039, 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 522, 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.039937044, 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000076507 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.671 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 459264 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.672 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 839, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 822, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 10166668, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 324281, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.102179329, 2024-08-01T04:27:18.673 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.034632111, 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.058888900, 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.017998393, 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 821, 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.121391318, 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000147857 2024-08-01T04:27:18.674 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 1627776, 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 139840 2024-08-01T04:27:18.675 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 1, 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-MDSRank": { 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-08-01T04:27:18.676 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 4135, 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.108757683, 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000026301 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: "finisher-PurgeQueue": { 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: "queue_len": 0, 2024-08-01T04:27:18.677 INFO:teuthology.orchestra.run.smithi163.stdout: "complete_latency": { 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": { 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: "request": 0, 2024-08-01T04:27:18.678 INFO:teuthology.orchestra.run.smithi163.stdout: "reply": 0, 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: "reply_latency": { 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_reply": 0, 2024-08-01T04:27:18.679 INFO:teuthology.orchestra.run.smithi163.stdout: "forward": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_complete": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_fetch_keys": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_commit": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_split": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_merge": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes": 1586, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_top": 0, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_bottom": 1586, 2024-08-01T04:27:18.680 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pin_tail": 0, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_pinned": 1586, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_expired": 0, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "inodes_with_caps": 0, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": 0, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "subtrees": 6, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse": 0, 2024-08-01T04:27:18.681 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_hit": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_forward": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_discover": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_dir_fetch": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_remote_ino": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "traverse_lock": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "load_cent": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "q": 0, 2024-08-01T04:27:18.682 INFO:teuthology.orchestra.run.smithi163.stdout: "exported": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "exported_inodes": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "imported": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "imported_inodes": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_dir_fetch": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_backtrace_fetch": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "openino_peer_discover": 0, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rfiles": 325, 2024-08-01T04:27:18.683 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rbytes": 13424230508, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "root_rsnaps": 6, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_fetch": 0, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_set_tag": 0, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_backtrace_repaired": 0, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_inotable_repaired": 0, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_inodes": 0, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dir_base_inodes": 0, 2024-08-01T04:27:18.684 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_dirfrag_rstats": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "scrub_file_inodes": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_inode_file_caps": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_revoke": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_grant": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_trunc": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-01T04:27:18.685 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps": 0, 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_caps_dirty": 0, 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_cap_release": 0, 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "process_request_cap_release": 0 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_cache": { 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update": 0, 2024-08-01T04:27:18.686 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_update_receipt": 0, 2024-08-01T04:27:18.687 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_try_discover": 0, 2024-08-01T04:27:18.690 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_send_discover": 0, 2024-08-01T04:27:18.690 INFO:teuthology.orchestra.run.smithi163.stdout: "dir_handle_discover": 0, 2024-08-01T04:27:18.690 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays": 0, 2024-08-01T04:27:18.690 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_delayed": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "num_strays_enqueuing": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_created": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_enqueued": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_reintegrated": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "strays_migrated": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_processing": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_enqueued": 0, 2024-08-01T04:27:18.691 INFO:teuthology.orchestra.run.smithi163.stdout: "num_recovering_prioritized": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_started": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "recovery_completed": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_path": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_quiesce_inode": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_enqueue_scrub": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_exportdir": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_flush": 0, 2024-08-01T04:27:18.692 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragmentdir": 0, 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_fragstats": 0, 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "ireq_inodestats": 0 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_log": { 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "evlrg": 0, 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "evadd": 0, 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "evex": 0, 2024-08-01T04:27:18.693 INFO:teuthology.orchestra.run.smithi163.stdout: "evtrm": 0, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "ev": 14242, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "evexg": 0, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "evexd": 0, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "segadd": 0, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "segex": 0, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "segtrm": 0, 2024-08-01T04:27:18.694 INFO:teuthology.orchestra.run.smithi163.stdout: "seg": 21, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "segmjr": 3, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "segexg": 0, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "segexd": 0, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "expos": 4194304, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "wrpos": 56696814, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "rdpos": 56696702, 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "jlat": { 2024-08-01T04:27:18.695 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: "replayed": 14242 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_mem": { 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: "ino": 926, 2024-08-01T04:27:18.696 INFO:teuthology.orchestra.run.smithi163.stdout: "ino+": 1399, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "ino-": 473, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "dir": 108, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "dir+": 126, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "dir-": 18, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "dn": 1586, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "dn+": 1642, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "dn-": 56, 2024-08-01T04:27:18.697 INFO:teuthology.orchestra.run.smithi163.stdout: "cap": 0, 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: "cap+": 0, 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: "cap-": 0, 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: "rss": 55288, 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: "heap": 215328 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_server": { 2024-08-01T04:27:18.698 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_client_request": 0, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "dispatch_server_request": 0, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_request": 0, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_client_session": 0, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "handle_peer_request": 0, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "req_create_latency": { 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.699 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getattr_latency": { 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getfilelock_latency": { 2024-08-01T04:27:18.700 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: "req_link_latency": { 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.701 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookup_latency": { 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookuphash_latency": { 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.702 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupino_latency": { 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.703 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupname_latency": { 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupparent_latency": { 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.704 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lookupsnap_latency": { 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.705 INFO:teuthology.orchestra.run.smithi163.stdout: "req_lssnap_latency": { 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mkdir_latency": { 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.706 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mknod_latency": { 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.707 INFO:teuthology.orchestra.run.smithi163.stdout: "req_mksnap_latency": { 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "req_open_latency": { 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.708 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: "req_readdir_latency": { 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rename_latency": { 2024-08-01T04:27:18.709 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: "req_renamesnap_latency": { 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.710 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "req_snapdiff_latency": { 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmdir_latency": { 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.711 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmsnap_latency": { 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "req_rmxattr_latency": { 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.712 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setattr_latency": { 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.713 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setdirlayout_latency": { 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setfilelock_latency": { 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.714 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setlayout_latency": { 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: "req_setxattr_latency": { 2024-08-01T04:27:18.715 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: "req_symlink_latency": { 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.716 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "req_unlink_latency": { 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_revoke_eviction": 0, 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "cap_acquisition_throttle": 0, 2024-08-01T04:27:18.717 INFO:teuthology.orchestra.run.smithi163.stdout: "req_getvxattr_latency": { 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_sessions": { 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: "session_count": 2, 2024-08-01T04:27:18.718 INFO:teuthology.orchestra.run.smithi163.stdout: "session_add": 5, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "session_remove": 3, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_open": 4, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "sessions_stale": 0, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "total_load": 0, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "average_load": 0, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "avg_session_uptime": 3636, 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: "mdthresh_evicted": 0 2024-08-01T04:27:18.719 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:18.720 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:18.721 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:18.721 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:18.721 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:18.721 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:18.721 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:18.722 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:18.723 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:18.724 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:18.724 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:18.724 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:18.724 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:18.724 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-08-01T04:27:18.724 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 251839, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 1077, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:18.725 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 8632, 2024-08-01T04:27:18.726 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 190, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 3502912, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 38206, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:18.727 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "objecter": { 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "op_active": 0, 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "op_laggy": 0, 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send": 2490, 2024-08-01T04:27:18.728 INFO:teuthology.orchestra.run.smithi163.stdout: "op_send_bytes": 26, 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "op_resend": 4, 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "op_reply": 2490, 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "op_latency": { 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2490, 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 40.889104621, 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.016421327 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.729 INFO:teuthology.orchestra.run.smithi163.stdout: "op_inflight": 0, 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "oplen_avg": { 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 2490, 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 2491 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "op": 2490, 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "op_r": 2490, 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "op_w": 0, 2024-08-01T04:27:18.730 INFO:teuthology.orchestra.run.smithi163.stdout: "op_rmw": 0, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "op_pg": 0, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_stat": 1498, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_create": 0, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_read": 989, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_write": 0, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writefull": 0, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_writesame": 0, 2024-08-01T04:27:18.731 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_append": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_zero": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_truncate": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_delete": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_mapext": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_sparse_read": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_clonerange": 0, 2024-08-01T04:27:18.732 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_getxattr": 2, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_setxattr": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_cmpxattr": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_rmxattr": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_resetxattrs": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_call": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_watch": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_notify": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_src_cmpxattr": 0, 2024-08-01T04:27:18.733 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_pgls_filter": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "osdop_other": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_active": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_send": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_resend": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "linger_ping": 0, 2024-08-01T04:27:18.734 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_active": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_send": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "poolop_resend": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_active": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_send": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "poolstat_resend": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_active": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_send": 0, 2024-08-01T04:27:18.735 INFO:teuthology.orchestra.run.smithi163.stdout: "statfs_resend": 0, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "command_active": 0, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "command_send": 0, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "command_resend": 0, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "map_epoch": 100, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "map_full": 0, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "map_inc": 14, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_sessions": 8, 2024-08-01T04:27:18.736 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_open": 8, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_session_close": 0, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_laggy": 0, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_wr": 0, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_rd": 2, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_del": 0 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "oft": { 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_objs": 0, 2024-08-01T04:27:18.737 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_kv_pairs": 0, 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_updates": 0, 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: "omap_total_removes": 0 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: "osd-slow-ops": { 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: "slow_ops_count": 0 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.738 INFO:teuthology.orchestra.run.smithi163.stdout: "purge_queue": { 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops": 0, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_ops_high_water": 0, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing": 0, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executing_high_water": 0, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed_ops": 0, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_executed": 0, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "pq_item_in_journal": 0 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.739 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 2751, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 54590953, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 2751, 2024-08-01T04:27:18.740 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2751, 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 54590953, 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.741 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_bytes": { 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:18.742 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 2490, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 52528697, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2490, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 52528697, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.743 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-objecter_ops": { 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 1024, 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:18.744 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 2490, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 2490, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 2490, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 2490, 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:18.745 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle": { 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-08-01T04:27:18.746 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-08-01T04:27:18.747 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-write_buf_throttle-0x55b11a0a6b40": { 2024-08-01T04:27:18.748 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 3758096384, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 0, 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 0, 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.750 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:18.751 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-08-01T04:27:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: pgmap v343: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 KiB/s rd, 181 B/s wr, 6 op/s 2024-08-01T04:27:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:20.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:20.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: osdmap e101: 12 total, 12 up, 12 in 2024-08-01T04:27:20.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:19 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.232 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: pgmap v343: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 KiB/s rd, 181 B/s wr, 6 op/s 2024-08-01T04:27:20.232 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:20.232 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:20.232 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: osdmap e101: 12 total, 12 up, 12 in 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.233 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:19 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.265 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-08-01T04:27:20.266 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-08-01T04:27:20.266 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-01T04:27:20.266 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-08-01T04:27:20.266 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-08-01T04:27:20.267 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-08-01T04:27:20.267 DEBUG:tasks.check_counter:Getting stats from l 2024-08-01T04:27:20.267 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-01T04:27:20.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: pgmap v343: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 KiB/s rd, 181 B/s wr, 6 op/s 2024-08-01T04:27:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: osdmap e101: 12 total, 12 up, 12 in 2024-08-01T04:27:20.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.300 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.301 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:19 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:20.882 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.c/config 2024-08-01T04:27:21.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:20 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-08-01T04:27:21.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:20 smithi163 ceph-mon[37383]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-01T04:27:21.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:20 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-08-01T04:27:21.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:20 smithi003 ceph-mon[32262]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-01T04:27:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:20 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-08-01T04:27:21.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:20 smithi160 ceph-mon[37224]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-01T04:27:22.091 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:21 smithi163 ceph-mon[37383]: pgmap v345: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:22.091 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:22.091 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:21 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:22.116 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-0": { 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 0, 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 2, 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 0, 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 0, 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 4, 2024-08-01T04:27:22.117 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 0, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.001011565, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.000000000, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.000193975, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:22.118 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.119 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 0 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-1": { 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 6, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 178, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 3344, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 188048, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 5, 2024-08-01T04:27:22.120 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 3, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.013185163, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.007934229, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.003189461, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 176, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.030258529, 2024-08-01T04:27:22.121 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000171923 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.122 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 188048 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "AsyncMessenger::Worker-2": { 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_messages": 240, 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages": 229, 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_bytes": 1585824, 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_bytes": 140864, 2024-08-01T04:27:22.123 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_created_connections": 2, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_active_connections": 1, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_total_time": 0.046098209, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_send_time": 0.015774812, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_recv_time": 0.044086802, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_messages_queue_lat": { 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 228, 2024-08-01T04:27:22.124 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.014860331, 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000065176 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_handle_ack_lat": { 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_recv_encrypted_bytes": 1585824, 2024-08-01T04:27:22.125 INFO:teuthology.orchestra.run.smithi163.stdout: "msgr_send_encrypted_bytes": 140864 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: "cct": { 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: "total_workers": 0, 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: "unhealthy_workers": 0 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: "mempool": { 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_bytes": 0, 2024-08-01T04:27:22.126 INFO:teuthology.orchestra.run.smithi163.stdout: "bloom_filter_items": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_bytes": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_alloc_items": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_bytes": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_data_items": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_onode_items": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-01T04:27:22.127 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_meta_items": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_bytes": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_other_items": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_cache_buffer_items": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_bytes": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_extent_items": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_bytes": 0, 2024-08-01T04:27:22.128 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_blob_items": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_shared_blob_items": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_inline_bl_items": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_bytes": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_fsck_items": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_bytes": 0, 2024-08-01T04:27:22.129 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_txc_items": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_deferred_items": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_bytes": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluestore_writing_items": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_bytes": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_items": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_bytes": 0, 2024-08-01T04:27:22.130 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_reader_items": 0, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_bytes": 0, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "bluefs_file_writer_items": 0, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_bytes": 148692, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_anon_items": 30, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_bytes": 0, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "buffer_meta_items": 0, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_bytes": 0, 2024-08-01T04:27:22.131 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_items": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_bytes": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_mapbl_items": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_bytes": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osd_pglog_items": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_bytes": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_items": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_bytes": 0, 2024-08-01T04:27:22.132 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_mapping_items": 0, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_bytes": 0, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "pgmap_items": 0, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_bytes": 928, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "mds_co_items": 2, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_bytes": 0, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_1_items": 0, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_bytes": 0, 2024-08-01T04:27:22.133 INFO:teuthology.orchestra.run.smithi163.stdout: "unittest_2_items": 0 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "val": 0, 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "max": 104857600, 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "get_started": 0, 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "get": 242, 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "get_sum": 1552000, 2024-08-01T04:27:22.134 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_fail": 0, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "get_or_fail_success": 242, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "take": 0, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "take_sum": 0, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "put": 242, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "put_sum": 1552000, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "wait": { 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "avgcount": 0, 2024-08-01T04:27:22.135 INFO:teuthology.orchestra.run.smithi163.stdout: "sum": 0.000000000, 2024-08-01T04:27:22.136 INFO:teuthology.orchestra.run.smithi163.stdout: "avgtime": 0.000000000 2024-08-01T04:27:22.136 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:22.136 INFO:teuthology.orchestra.run.smithi163.stdout: } 2024-08-01T04:27:22.136 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-08-01T04:27:22.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:21 smithi003 ceph-mon[32262]: pgmap v345: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:22.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:21 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:22.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:21 smithi160 ceph-mon[37224]: pgmap v345: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:22.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:22.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:21 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:22.792 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-01T04:27:22.793 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-01T04:27:22.793 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-08-01T04:27:22.793 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-08-01T04:27:22.793 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-08-01T04:27:22.793 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-08-01T04:27:22.794 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-01T04:27:22.805 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-01T04:27:22.806 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:27:22.806 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-01T04:27:22.876 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-08-01T04:27:22.876 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-01T04:27:22.934 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd blocklist ls 2024-08-01T04:27:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:22 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:22 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:22 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:24.130 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi003 to 177.5M 2024-08-01T04:27:24.130 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: Unable to set osd_memory_target on smithi003 to 186154393: error parsing value: Value '186154393' is below minimum 939524096 2024-08-01T04:27:24.130 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: pgmap v346: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:24.130 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.130 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.131 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:27:24.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:27:24.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:23 smithi003 ceph-mon[32262]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 -- 172.21.15.3:0/3025448063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c109900 msgr2=0x7fe40c109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 --2- 172.21.15.3:0/3025448063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c109900 0x7fe40c109ce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe400009930 tx=0x7fe40002a420 comp rx=0 tx=0).stop 2024-08-01T04:27:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 -- 172.21.15.3:0/3025448063 shutdown_connections 2024-08-01T04:27:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 --2- 172.21.15.3:0/3025448063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c1042a0 0x7fe40c1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 --2- 172.21.15.3:0/3025448063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c103900 0x7fe40c103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 --2- 172.21.15.3:0/3025448063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c109900 0x7fe40c109ce0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 -- 172.21.15.3:0/3025448063 >> 172.21.15.3:0/3025448063 conn(0x7fe40c0ff640 msgr2=0x7fe40c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 -- 172.21.15.3:0/3025448063 shutdown_connections 2024-08-01T04:27:24.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.135+0000 7fe413000640 1 -- 172.21.15.3:0/3025448063 wait complete. 2024-08-01T04:27:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.136+0000 7fe413000640 1 Processor -- start 2024-08-01T04:27:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.136+0000 7fe413000640 1 -- start start 2024-08-01T04:27:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe413000640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c103900 0x7fe40c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:24.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe413000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c1042a0 0x7fe40c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe413000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c109900 0x7fe40c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe413000640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c113d50 con 0x7fe40c1042a0 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe413000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c113bd0 con 0x7fe40c109900 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe413000640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c113ed0 con 0x7fe40c103900 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe411200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c1042a0 0x7fe40c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe411c00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c103900 0x7fe40c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe411200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c1042a0 0x7fe40c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50900/0 (socket says 172.21.15.3:50900) 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe411200640 1 -- 172.21.15.3:0/1871319321 learned_addr learned my addr 172.21.15.3:0/1871319321 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.137+0000 7fe412600640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c109900 0x7fe40c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe411c00640 1 -- 172.21.15.3:0/1871319321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c109900 msgr2=0x7fe40c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe411c00640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c109900 0x7fe40c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe411c00640 1 -- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c1042a0 msgr2=0x7fe40c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe411c00640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c1042a0 0x7fe40c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe411c00640 1 -- 172.21.15.3:0/1871319321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c19f1a0 con 0x7fe40c103900 2024-08-01T04:27:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe412600640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c109900 0x7fe40c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:27:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.138+0000 7fe411c00640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c103900 0x7fe40c19a1f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe40002a940 tx=0x7fe400003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:24.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.139+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40003f070 con 0x7fe40c103900 2024-08-01T04:27:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.139+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe400031c90 con 0x7fe40c103900 2024-08-01T04:27:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.139+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c19f430 con 0x7fe40c103900 2024-08-01T04:27:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.139+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400035030 con 0x7fe40c103900 2024-08-01T04:27:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.139+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c1a6d70 con 0x7fe40c103900 2024-08-01T04:27:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.140+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c105040 con 0x7fe40c103900 2024-08-01T04:27:24.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.142+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe400042070 con 0x7fe40c103900 2024-08-01T04:27:24.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.143+0000 7fe40a000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe3f40a2cf0 0x7fe3f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:24.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.143+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7fe4000efdc0 con 0x7fe40c103900 2024-08-01T04:27:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.144+0000 7fe411200640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe3f40a2cf0 0x7fe3f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:24.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.144+0000 7fe411200640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe3f40a2cf0 0x7fe3f40a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe40c101c20 tx=0x7fe3f8005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:24.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.147+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe4000bc2c0 con 0x7fe40c103900 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi003 to 177.5M 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: Unable to set osd_memory_target on smithi003 to 186154393: error parsing value: Value '186154393' is below minimum 939524096 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: pgmap v346: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.297 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:27:24.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:27:24.298 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:23 smithi163 ceph-mon[37383]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.359+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe40c103d60 con 0x7fe40c103900 2024-08-01T04:27:24.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.359+0000 7fe40a000640 1 -- 172.21.15.3:0/1871319321 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v101) ==== 81+0+928 (secure 0 0 0) 0x7fe4000c05c0 con 0x7fe40c103900 2024-08-01T04:27:24.361 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1348130449 2024-08-02T04:16:15.828376+0000 2024-08-01T04:27:24.361 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/346159818 2024-08-02T04:16:15.828376+0000 2024-08-01T04:27:24.361 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/320273213 2024-08-02T04:06:17.351564+0000 2024-08-01T04:27:24.361 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1530825959 2024-08-02T04:06:17.351564+0000 2024-08-01T04:27:24.361 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1579863007 2024-08-02T04:06:17.351564+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4015366337 2024-08-02T04:16:15.828376+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2048633599 2024-08-02T04:06:17.351564+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1530825959 2024-08-02T04:06:17.351564+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2792822839 2024-08-02T04:06:48.053573+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4012679146 2024-08-02T04:06:48.053573+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1617846146 2024-08-02T04:16:15.828376+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2156560379 2024-08-02T04:06:48.053573+0000 2024-08-01T04:27:24.362 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1368241138 2024-08-02T04:06:48.053573+0000 2024-08-01T04:27:24.363 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2309230633 2024-08-02T04:16:15.828376+0000 2024-08-01T04:27:24.363 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1617846146 2024-08-02T04:16:15.828376+0000 2024-08-01T04:27:24.363 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2156560379 2024-08-02T04:06:48.053573+0000 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:listed 16 entries 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe3f40a2cf0 msgr2=0x7fe3f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe3f40a2cf0 0x7fe3f40a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe40c101c20 tx=0x7fe3f8005f90 comp rx=0 tx=0).stop 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c103900 msgr2=0x7fe40c19a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c103900 0x7fe40c19a1f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe40002a940 tx=0x7fe400003b20 comp rx=0 tx=0).stop 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 shutdown_connections 2024-08-01T04:27:24.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe40c109900 0x7fe40c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c1042a0 0x7fe40c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fe3f40a2cf0 0x7fe3f40a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 --2- 172.21.15.3:0/1871319321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40c103900 0x7fe40c19a1f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:24.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.362+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 >> 172.21.15.3:0/1871319321 conn(0x7fe40c0ff640 msgr2=0x7fe40c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:24.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.363+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 shutdown_connections 2024-08-01T04:27:24.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:24.363+0000 7fe413000640 1 -- 172.21.15.3:0/1871319321 wait complete. 2024-08-01T04:27:24.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi003 to 177.5M 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: Unable to set osd_memory_target on smithi003 to 186154393: error parsing value: Value '186154393' is below minimum 939524096 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: pgmap v346: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-01T04:27:24.404 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:23 smithi160 ceph-mon[37224]: from='mgr.14928 ' entity='mgr.x' 2024-08-01T04:27:25.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:24 smithi003 ceph-mon[32262]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-08-01T04:27:25.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:24 smithi003 ceph-mon[32262]: Unable to set osd_memory_target on smithi163 to 184780288: error parsing value: Value '184780288' is below minimum 939524096 2024-08-01T04:27:25.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:25 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1871319321' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-01T04:27:25.288 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-01T04:27:25.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:27:25.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-01T04:27:25.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:24 smithi163 ceph-mon[37383]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-08-01T04:27:25.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:25 smithi163 ceph-mon[37383]: Unable to set osd_memory_target on smithi163 to 184780288: error parsing value: Value '184780288' is below minimum 939524096 2024-08-01T04:27:25.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:25 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1871319321' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-01T04:27:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:24 smithi160 ceph-mon[37224]: Adjusting osd_memory_target on smithi163 to 176.2M 2024-08-01T04:27:25.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:24 smithi160 ceph-mon[37224]: Unable to set osd_memory_target on smithi163 to 184780288: error parsing value: Value '184780288' is below minimum 939524096 2024-08-01T04:27:25.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:25 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1871319321' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-01T04:27:26.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:26 smithi163 ceph-mon[37383]: pgmap v347: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:26.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:26 smithi003 ceph-mon[32262]: pgmap v347: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:26.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:26 smithi160 ceph-mon[37224]: pgmap v347: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-08-01T04:27:26.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:27:26.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:27:26.556 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-01T04:27:26.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:27:26.645 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-08-01T04:27:26.757 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-01T04:27:26.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:27:26.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-01T04:27:26.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-01T04:27:26.820 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-08-01T04:27:27.207 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-01T04:27:27.220 INFO:tasks.cephadm:Teardown begin 2024-08-01T04:27:27.220 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-01T04:27:27.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:27:27.661 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:28.068 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:28 smithi003 ceph-mon[32262]: pgmap v348: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:27:28.295 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:28 smithi163 ceph-mon[37383]: pgmap v348: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:27:28.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:28 smithi160 ceph-mon[37224]: pgmap v348: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-01T04:27:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 -- 172.21.15.3:0/3031128381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638076d70 msgr2=0x7f8638112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 --2- 172.21.15.3:0/3031128381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638076d70 0x7f8638112060 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8624009900 tx=0x7f862402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 -- 172.21.15.3:0/3031128381 shutdown_connections 2024-08-01T04:27:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 --2- 172.21.15.3:0/3031128381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86381125a0 0x7f8638114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 --2- 172.21.15.3:0/3031128381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8638076d70 0x7f8638112060 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 --2- 172.21.15.3:0/3031128381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f8638076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 -- 172.21.15.3:0/3031128381 >> 172.21.15.3:0/3031128381 conn(0x7f86380ff600 msgr2=0x7f8638101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.457+0000 7f863d200640 1 -- 172.21.15.3:0/3031128381 shutdown_connections 2024-08-01T04:27:28.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.458+0000 7f863d200640 1 -- 172.21.15.3:0/3031128381 wait complete. 2024-08-01T04:27:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.459+0000 7f863d200640 1 Processor -- start 2024-08-01T04:27:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.459+0000 7f863d200640 1 -- start start 2024-08-01T04:27:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f863d200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f863819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f863d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8638076d70 0x7f863819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f863d200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86381125a0 0x7f86381a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:28.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f863d200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8638118180 con 0x7f86381125a0 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f863d200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8638118000 con 0x7f8638076d70 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f863d200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f8638118300 con 0x7f8638076450 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f8637400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f863819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f8637400640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f863819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.3:45704/0 (socket says 172.21.15.3:45704) 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f8637400640 1 -- 172.21.15.3:0/123738992 learned_addr learned my addr 172.21.15.3:0/123738992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f8636a00640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8638076d70 0x7f863819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:28.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.460+0000 7f8637e00640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86381125a0 0x7f86381a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f8637400640 1 -- 172.21.15.3:0/123738992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8638076d70 msgr2=0x7f863819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f8637400640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8638076d70 0x7f863819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f8637400640 1 -- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86381125a0 msgr2=0x7f86381a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f8637400640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86381125a0 0x7f86381a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f8637400640 1 -- 172.21.15.3:0/123738992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381a35e0 con 0x7f8638076450 2024-08-01T04:27:28.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f8637400640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f863819e630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f862800e9f0 tx=0x7f862800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862800cd80 con 0x7f8638076450 2024-08-01T04:27:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8628005500 con 0x7f8638076450 2024-08-01T04:27:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381a38d0 con 0x7f8638076450 2024-08-01T04:27:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.461+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628010690 con 0x7f8638076450 2024-08-01T04:27:28.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.462+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381ab1b0 con 0x7f8638076450 2024-08-01T04:27:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.463+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8638076830 con 0x7f8638076450 2024-08-01T04:27:28.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.465+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f86280046c0 con 0x7f8638076450 2024-08-01T04:27:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.466+0000 7f862fe00640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f861c0a2d70 0x7f861c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:28.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.466+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f86280cf860 con 0x7f8638076450 2024-08-01T04:27:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.467+0000 7f8636a00640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f861c0a2d70 0x7f861c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:28.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.467+0000 7f8636a00640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f861c0a2d70 0x7f861c0a5230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8624000c00 tx=0x7f8624005b50 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:28.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.469+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f862809c3b0 con 0x7f8638076450 2024-08-01T04:27:28.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.700+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f863819f990 con 0x7f8638076450 2024-08-01T04:27:28.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.701+0000 7f862fe00640 1 -- 172.21.15.3:0/123738992 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f863819f990 con 0x7f8638076450 2024-08-01T04:27:28.703 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:27:28.703 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:27:28.703 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:27:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.705+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f861c0a2d70 msgr2=0x7f861c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.705+0000 7f863d200640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f861c0a2d70 0x7f861c0a5230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8624000c00 tx=0x7f8624005b50 comp rx=0 tx=0).stop 2024-08-01T04:27:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.705+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 msgr2=0x7f863819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:28.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.705+0000 7f863d200640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f863819e630 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f862800e9f0 tx=0x7f862800edf0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 shutdown_connections 2024-08-01T04:27:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86381125a0 0x7f86381a2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8638076d70 0x7f863819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f861c0a2d70 0x7f861c0a5230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 --2- 172.21.15.3:0/123738992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8638076450 0x7f863819e630 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 >> 172.21.15.3:0/123738992 conn(0x7f86380ff600 msgr2=0x7f86381019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 shutdown_connections 2024-08-01T04:27:28.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:28.706+0000 7f863d200640 1 -- 172.21.15.3:0/123738992 wait complete. 2024-08-01T04:27:29.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:29 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/123738992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:27:29.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:29 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/123738992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:27:29.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:29 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/123738992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:27:29.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs dump --format=json 2024-08-01T04:27:30.044 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:30.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:30 smithi163 ceph-mon[37383]: pgmap v349: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-08-01T04:27:30.299 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:30 smithi003 ceph-mon[32262]: pgmap v349: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-08-01T04:27:30.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:30 smithi160 ceph-mon[37224]: pgmap v349: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-08-01T04:27:30.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.677+0000 7f7062e00640 1 -- 172.21.15.3:0/2300817223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c108180 msgr2=0x7f705c108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.677+0000 7f7062e00640 1 --2- 172.21.15.3:0/2300817223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c108180 0x7f705c108560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02a3f0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 -- 172.21.15.3:0/2300817223 shutdown_connections 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 --2- 172.21.15.3:0/2300817223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c10d260 0x7f705c114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 --2- 172.21.15.3:0/2300817223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 0x7f705c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 --2- 172.21.15.3:0/2300817223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c108180 0x7f705c108560 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 -- 172.21.15.3:0/2300817223 >> 172.21.15.3:0/2300817223 conn(0x7f705c0778d0 msgr2=0x7f705c079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:30.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 -- 172.21.15.3:0/2300817223 shutdown_connections 2024-08-01T04:27:30.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.678+0000 7f7062e00640 1 -- 172.21.15.3:0/2300817223 wait complete. 2024-08-01T04:27:30.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.679+0000 7f7062e00640 1 Processor -- start 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.679+0000 7f7062e00640 1 -- start start 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7062e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 0x7f705c1a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7062e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 0x7f705c1a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7062e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c10d260 0x7f705c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7062e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1182d0 con 0x7f705c108180 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7062e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f705c118150 con 0x7f705c10d260 2024-08-01T04:27:30.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7062e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f705c118450 con 0x7f705c108b30 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f705be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 0x7f705c1a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f705be00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 0x7f705c1a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50938/0 (socket says 172.21.15.3:50938) 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f705be00640 1 -- 172.21.15.3:0/3833554820 learned_addr learned my addr 172.21.15.3:0/3833554820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.680+0000 7f7053e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 0x7f705c1a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f7060e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c10d260 0x7f705c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f705be00640 1 -- 172.21.15.3:0/3833554820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 msgr2=0x7f705c1a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:30.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f705be00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 0x7f705c1a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f705be00640 1 -- 172.21.15.3:0/3833554820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c10d260 msgr2=0x7f705c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:30.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f705be00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c10d260 0x7f705c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f705be00640 1 -- 172.21.15.3:0/3833554820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c19eed0 con 0x7f705c108180 2024-08-01T04:27:30.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.681+0000 7f7053e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 0x7f705c1a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:27:30.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.682+0000 7f705be00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 0x7f705c1a4590 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f704c004770 tx=0x7f704c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:30.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.682+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c03f070 con 0x7f705c108180 2024-08-01T04:27:30.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.682+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c031ca0 con 0x7f705c108180 2024-08-01T04:27:30.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.682+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c19f160 con 0x7f705c108180 2024-08-01T04:27:30.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.682+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c035030 con 0x7f705c108180 2024-08-01T04:27:30.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.682+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1ab340 con 0x7f705c108180 2024-08-01T04:27:30.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.684+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c109610 con 0x7f705c108180 2024-08-01T04:27:30.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.686+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f704c042070 con 0x7f705c108180 2024-08-01T04:27:30.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.688+0000 7f7059600640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70400a2cf0 0x7f70400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:30.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.689+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f704c0efba0 con 0x7f705c108180 2024-08-01T04:27:30.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.689+0000 7f7053e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70400a2cf0 0x7f70400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:30.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.690+0000 7f7053e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70400a2cf0 0x7f70400a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f705c19feb0 tx=0x7f7044005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:30.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.694+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f704c0bc0a0 con 0x7f705c108180 2024-08-01T04:27:30.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.937+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f705c110440 con 0x7f705c108180 2024-08-01T04:27:30.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.939+0000 7f7059600640 1 -- 172.21.15.3:0/3833554820 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11908 (secure 0 0 0) 0x7f704c0c03a0 con 0x7f705c108180 2024-08-01T04:27:30.943 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:27:30.943 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-08-01T04:27:30.944 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-08-01T04:20:37:072326+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/367445798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":367445798},{"type":"v1","addr":"172.21.15.3:6839","nonce":367445798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/4230398308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4230398308},{"type":"v1","addr":"172.21.15.163:6833","nonce":4230398308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/3012672686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":3012672686},{"type":"v1","addr":"172.21.15.160:6833","nonce":3012672686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3318532700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3318532700},{"type":"v1","addr":"172.21.15.3:6835","nonce":3318532700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2143574012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2143574012},{"type":"v1","addr":"172.21.15.163:6835","nonce":2143574012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/258713238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":258713238},{"type":"v1","addr":"172.21.15.3:6837","nonce":258713238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-01T04:13:55.770404+0000","modified":"2024-08-01T04:20:36.490638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24548,"mds_1":24520,"mds_2":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/3179303046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3179303046},{"type":"v1","addr":"172.21.15.160:6835","nonce":3179303046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6837/1698078935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":1698078935},{"type":"v1","addr":"172.21.15.163:6837","nonce":1698078935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.163:6839/4057809310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6838","nonce":4057809310},{"type":"v1","addr":"172.21.15.163:6839","nonce":4057809310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.160:6837/3127569351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3127569351},{"type":"v1","addr":"172.21.15.160:6837","nonce":3127569351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.3:6841/2435488646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":2435488646},{"type":"v1","addr":"172.21.15.3:6841","nonce":2435488646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.160:6839/1741438883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":1741438883},{"type":"v1","addr":"172.21.15.160:6839","nonce":1741438883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24548,"qdb_cluster":[24548,24520,24506]},"id":1}]} 2024-08-01T04:27:30.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70400a2cf0 msgr2=0x7f70400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70400a2cf0 0x7f70400a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f705c19feb0 tx=0x7f7044005f90 comp rx=0 tx=0).stop 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 msgr2=0x7f705c1a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 0x7f705c1a4590 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f704c004770 tx=0x7f704c003c10 comp rx=0 tx=0).stop 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 shutdown_connections 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f705c10d260 0x7f705c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f70400a2cf0 0x7f70400a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705c108b30 0x7f705c1a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 --2- 172.21.15.3:0/3833554820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f705c108180 0x7f705c1a4590 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.942+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 >> 172.21.15.3:0/3833554820 conn(0x7f705c0778d0 msgr2=0x7f705c078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.943+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 shutdown_connections 2024-08-01T04:27:30.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:30.943+0000 7f7062e00640 1 -- 172.21.15.3:0/3833554820 wait complete. 2024-08-01T04:27:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:31 smithi163 ceph-mon[37383]: pgmap v350: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 482 B/s wr, 6 op/s 2024-08-01T04:27:31.296 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:31 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3833554820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:27:31.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:31 smithi003 ceph-mon[32262]: pgmap v350: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 482 B/s wr, 6 op/s 2024-08-01T04:27:31.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:31 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3833554820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:27:31.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:31 smithi160 ceph-mon[37224]: pgmap v350: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 482 B/s wr, 6 op/s 2024-08-01T04:27:31.403 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:31 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3833554820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-01T04:27:31.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd dump --format=json 2024-08-01T04:27:32.254 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:32.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 -- 172.21.15.3:0/2216451709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28076d70 msgr2=0x7f3e28112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:32.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 --2- 172.21.15.3:0/2216451709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28076d70 0x7f3e28112220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009900 tx=0x7f3e1002a3f0 comp rx=0 tx=0).stop 2024-08-01T04:27:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 -- 172.21.15.3:0/2216451709 shutdown_connections 2024-08-01T04:27:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 --2- 172.21.15.3:0/2216451709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28112760 0x7f3e28114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 --2- 172.21.15.3:0/2216451709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28076d70 0x7f3e28112220 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 --2- 172.21.15.3:0/2216451709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e28076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:32.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 -- 172.21.15.3:0/2216451709 >> 172.21.15.3:0/2216451709 conn(0x7f3e280ff7c0 msgr2=0x7f3e28101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.931+0000 7f3e2d600640 1 -- 172.21.15.3:0/2216451709 shutdown_connections 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.932+0000 7f3e2d600640 1 -- 172.21.15.3:0/2216451709 wait complete. 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 Processor -- start 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 -- start start 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e281a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28076d70 0x7f3e281a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:32.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28112760 0x7f3e2819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28118320 con 0x7f3e28112760 2024-08-01T04:27:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281181a0 con 0x7f3e28076450 2024-08-01T04:27:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.933+0000 7f3e2d600640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281184a0 con 0x7f3e28076d70 2024-08-01T04:27:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e281a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e281a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:54876/0 (socket says 172.21.15.3:54876) 2024-08-01T04:27:32.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 -- 172.21.15.3:0/3000074360 learned_addr learned my addr 172.21.15.3:0/3000074360 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e1e000640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28076d70 0x7f3e281a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e27400640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28112760 0x7f3e2819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 -- 172.21.15.3:0/3000074360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28076d70 msgr2=0x7f3e281a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28076d70 0x7f3e281a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 -- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28112760 msgr2=0x7f3e2819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28112760 0x7f3e2819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:32.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e26a00640 1 -- 172.21.15.3:0/3000074360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2819ece0 con 0x7f3e28076450 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e27400640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28112760 0x7f3e2819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.934+0000 7f3e1e000640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28076d70 0x7f3e281a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.935+0000 7f3e26a00640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e281a43f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800ebc0 tx=0x7f3e1800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.935+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18010630 con 0x7f3e28076450 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.935+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2819efd0 con 0x7f3e28076450 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.935+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e18010cb0 con 0x7f3e28076450 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.935+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18018bd0 con 0x7f3e28076450 2024-08-01T04:27:32.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.935+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e281ab2c0 con 0x7f3e28076450 2024-08-01T04:27:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.937+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e28076830 con 0x7f3e28076450 2024-08-01T04:27:32.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.939+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f3e180107d0 con 0x7f3e28076450 2024-08-01T04:27:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.940+0000 7f3e1fe00640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2cf0 0x7f3e0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.940+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f3e180cfb40 con 0x7f3e28076450 2024-08-01T04:27:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.940+0000 7f3e1e000640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2cf0 0x7f3e0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.941+0000 7f3e1e000640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2cf0 0x7f3e0c0a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e10000c00 tx=0x7f3e10031510 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:32.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:32.944+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3e1809c040 con 0x7f3e28076450 2024-08-01T04:27:33.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.143+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e281ab4f0 con 0x7f3e28076450 2024-08-01T04:27:33.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.146+0000 7f3e1fe00640 1 -- 172.21.15.3:0/3000074360 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+23271 (secure 0 0 0) 0x7f3e180a0340 con 0x7f3e28076450 2024-08-01T04:27:33.147 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-08-01T04:27:33.147 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":101,"fsid":"2f6d75a6-4fbb-11ef-bcc9-c7b262605968","created":"2024-08-01T04:05:46.214560+0000","modified":"2024-08-01T04:27:19.598472+0000","last_up_change":"2024-08-01T04:12:54.365711+0000","last_in_change":"2024-08-01T04:12:34.776090+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-01T04:09:04.723203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-01T04:13:50.125660+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-01T04:13:53.246715+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0fdfd41-e466-4275-8212-709184cca9a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6803","nonce":1021833457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6805","nonce":1021833457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6809","nonce":1021833457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":1021833457},{"type":"v1","addr":"172.21.15.3:6807","nonce":1021833457}]},"public_addr":"172.21.15.3:6803/1021833457","cluster_addr":"172.21.15.3:6805/1021833457","heartbeat_back_addr":"172.21.15.3:6809/1021833457","heartbeat_front_addr":"172.21.15.3:6807/1021833457","state":["exists","up"]},{"osd":1,"uuid":"f9b159fc-e987-4b22-b63e-910d2fc377e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6801","nonce":1858231238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6803","nonce":1858231238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6807","nonce":1858231238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":1858231238},{"type":"v1","addr":"172.21.15.160:6805","nonce":1858231238}]},"public_addr":"172.21.15.160:6801/1858231238","cluster_addr":"172.21.15.160:6803/1858231238","heartbeat_back_addr":"172.21.15.160:6807/1858231238","heartbeat_front_addr":"172.21.15.160:6805/1858231238","state":["exists","up"]},{"osd":2,"uuid":"90f0ad0d-8599-4670-868b-e0b0a593aa0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6801","nonce":2416358231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6803","nonce":2416358231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6807","nonce":2416358231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":2416358231},{"type":"v1","addr":"172.21.15.163:6805","nonce":2416358231}]},"public_addr":"172.21.15.163:6801/2416358231","cluster_addr":"172.21.15.163:6803/2416358231","heartbeat_back_addr":"172.21.15.163:6807/2416358231","heartbeat_front_addr":"172.21.15.163:6805/2416358231","state":["exists","up"]},{"osd":3,"uuid":"97d21a34-b1bd-4f38-ac45-28f4f9360b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6811","nonce":1408863124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6813","nonce":1408863124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6817","nonce":1408863124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":1408863124},{"type":"v1","addr":"172.21.15.3:6815","nonce":1408863124}]},"public_addr":"172.21.15.3:6811/1408863124","cluster_addr":"172.21.15.3:6813/1408863124","heartbeat_back_addr":"172.21.15.3:6817/1408863124","heartbeat_front_addr":"172.21.15.3:6815/1408863124","state":["exists","up"]},{"osd":4,"uuid":"7838d1fc-3355-4659-b61b-5fa5717d70b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6809","nonce":1218975759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6811","nonce":1218975759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6815","nonce":1218975759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1218975759},{"type":"v1","addr":"172.21.15.160:6813","nonce":1218975759}]},"public_addr":"172.21.15.160:6809/1218975759","cluster_addr":"172.21.15.160:6811/1218975759","heartbeat_back_addr":"172.21.15.160:6815/1218975759","heartbeat_front_addr":"172.21.15.160:6813/1218975759","state":["exists","up"]},{"osd":5,"uuid":"2f0ce098-7164-49a0-9d74-1c6afc6a32c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6809","nonce":1207930650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6811","nonce":1207930650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6815","nonce":1207930650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1207930650},{"type":"v1","addr":"172.21.15.163:6813","nonce":1207930650}]},"public_addr":"172.21.15.163:6809/1207930650","cluster_addr":"172.21.15.163:6811/1207930650","heartbeat_back_addr":"172.21.15.163:6815/1207930650","heartbeat_front_addr":"172.21.15.163:6813/1207930650","state":["exists","up"]},{"osd":6,"uuid":"b3de683d-970d-4e25-9574-48cf63d401ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6819","nonce":885876521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6821","nonce":885876521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6825","nonce":885876521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":885876521},{"type":"v1","addr":"172.21.15.3:6823","nonce":885876521}]},"public_addr":"172.21.15.3:6819/885876521","cluster_addr":"172.21.15.3:6821/885876521","heartbeat_back_addr":"172.21.15.3:6825/885876521","heartbeat_front_addr":"172.21.15.3:6823/885876521","state":["exists","up"]},{"osd":7,"uuid":"e2878577-9071-4b13-92ed-0cace6099cfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6817","nonce":3141457315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6819","nonce":3141457315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6823","nonce":3141457315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3141457315},{"type":"v1","addr":"172.21.15.160:6821","nonce":3141457315}]},"public_addr":"172.21.15.160:6817/3141457315","cluster_addr":"172.21.15.160:6819/3141457315","heartbeat_back_addr":"172.21.15.160:6823/3141457315","heartbeat_front_addr":"172.21.15.160:6821/3141457315","state":["exists","up"]},{"osd":8,"uuid":"c865f68f-ddd2-4332-bbfc-26b68a4b85b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6817","nonce":2678360734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6819","nonce":2678360734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6823","nonce":2678360734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":2678360734},{"type":"v1","addr":"172.21.15.163:6821","nonce":2678360734}]},"public_addr":"172.21.15.163:6817/2678360734","cluster_addr":"172.21.15.163:6819/2678360734","heartbeat_back_addr":"172.21.15.163:6823/2678360734","heartbeat_front_addr":"172.21.15.163:6821/2678360734","state":["exists","up"]},{"osd":9,"uuid":"f91a1874-e1c3-40ec-b6de-55b00c8d40e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6827","nonce":2204833975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6829","nonce":2204833975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6833","nonce":2204833975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":2204833975},{"type":"v1","addr":"172.21.15.3:6831","nonce":2204833975}]},"public_addr":"172.21.15.3:6827/2204833975","cluster_addr":"172.21.15.3:6829/2204833975","heartbeat_back_addr":"172.21.15.3:6833/2204833975","heartbeat_front_addr":"172.21.15.3:6831/2204833975","state":["exists","up"]},{"osd":10,"uuid":"64b32252-f40b-450c-bacc-c37ac7660b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6825","nonce":1295991046}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6827","nonce":1295991046}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6831","nonce":1295991046}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":1295991046},{"type":"v1","addr":"172.21.15.160:6829","nonce":1295991046}]},"public_addr":"172.21.15.160:6825/1295991046","cluster_addr":"172.21.15.160:6827/1295991046","heartbeat_back_addr":"172.21.15.160:6831/1295991046","heartbeat_front_addr":"172.21.15.160:6829/1295991046","state":["exists","up"]},{"osd":11,"uuid":"e4ed8cac-abd1-4c00-ac25-43f5b5e19bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6825","nonce":2977436589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6827","nonce":2977436589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6831","nonce":2977436589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2977436589},{"type":"v1","addr":"172.21.15.163:6829","nonce":2977436589}]},"public_addr":"172.21.15.163:6825/2977436589","cluster_addr":"172.21.15.163:6827/2977436589","heartbeat_back_addr":"172.21.15.163:6831/2977436589","heartbeat_front_addr":"172.21.15.163:6829/2977436589","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:19.439631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:08:40.742525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:02.181166+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:32.113552+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:09:55.091261+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:17.582596+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:10:48.545573+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:11.928289+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:11:34.454466+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:05.338134+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:29.274566+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-01T04:12:52.432652+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/1348130449":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/346159818":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/320273213":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6801/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/1579863007":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/4015366337":"2024-08-02T04:16:15.828376+0000","172.21.15.3:0/2048633599":"2024-08-02T04:06:17.351564+0000","172.21.15.3:6800/1530825959":"2024-08-02T04:06:17.351564+0000","172.21.15.3:0/2792822839":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/4012679146":"2024-08-02T04:06:48.053573+0000","172.21.15.3:6800/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6800/2156560379":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/1368241138":"2024-08-02T04:06:48.053573+0000","172.21.15.3:0/2309230633":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/1617846146":"2024-08-02T04:16:15.828376+0000","172.21.15.3:6801/2156560379":"2024-08-02T04:06:48.053573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-01T04:27:33.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.148+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2cf0 msgr2=0x7f3e0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:33.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.148+0000 7f3e2d600640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2cf0 0x7f3e0c0a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3e10000c00 tx=0x7f3e10031510 comp rx=0 tx=0).stop 2024-08-01T04:27:33.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.148+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 msgr2=0x7f3e281a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:33.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.148+0000 7f3e2d600640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e281a43f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800ebc0 tx=0x7f3e1800c6a0 comp rx=0 tx=0).stop 2024-08-01T04:27:33.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 shutdown_connections 2024-08-01T04:27:33.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e28112760 0x7f3e2819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:33.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f3e0c0a2cf0 0x7f3e0c0a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:33.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e28076d70 0x7f3e281a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:33.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 --2- 172.21.15.3:0/3000074360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e28076450 0x7f3e281a43f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:33.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 >> 172.21.15.3:0/3000074360 conn(0x7f3e280ff7c0 msgr2=0x7f3e28101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:33.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 shutdown_connections 2024-08-01T04:27:33.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:33.149+0000 7f3e2d600640 1 -- 172.21.15.3:0/3000074360 wait complete. 2024-08-01T04:27:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:33 smithi163 ceph-mon[37383]: pgmap v351: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-01T04:27:33.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:33 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3000074360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:27:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:33 smithi003 ceph-mon[32262]: pgmap v351: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-01T04:27:33.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:33 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3000074360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:27:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:33 smithi160 ceph-mon[37224]: pgmap v351: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-01T04:27:33.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:33 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3000074360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-01T04:27:34.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs fail cephfs 2024-08-01T04:27:34.759 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:35.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 -- 172.21.15.3:0/4190817633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076450 msgr2=0x7fbb34076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:35.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/4190817633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076450 0x7fbb34076830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009900 tx=0x7fbb2402a3f0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 -- 172.21.15.3:0/4190817633 shutdown_connections 2024-08-01T04:27:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/4190817633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34112760 0x7fbb34114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/4190817633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb34076d70 0x7fbb34112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/4190817633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076450 0x7fbb34076830 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 -- 172.21.15.3:0/4190817633 >> 172.21.15.3:0/4190817633 conn(0x7fbb340ff7a0 msgr2=0x7fbb34101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 -- 172.21.15.3:0/4190817633 shutdown_connections 2024-08-01T04:27:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.444+0000 7fbb3aa00640 1 -- 172.21.15.3:0/4190817633 wait complete. 2024-08-01T04:27:35.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:35 smithi003 ceph-mon[32262]: pgmap v352: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-01T04:27:35.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.445+0000 7fbb3aa00640 1 Processor -- start 2024-08-01T04:27:35.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 -- start start 2024-08-01T04:27:35.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34076450 0x7fbb341a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076d70 0x7fbb341a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb34112760 0x7fbb3419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb341182f0 con 0x7fbb34076d70 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34118170 con 0x7fbb34076450 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.446+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34118470 con 0x7fbb34112760 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34076450 0x7fbb341a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:35.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34076450 0x7fbb341a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:54906/0 (socket says 172.21.15.3:54906) 2024-08-01T04:27:35.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33e00640 1 -- 172.21.15.3:0/1619001590 learned_addr learned my addr 172.21.15.3:0/1619001590 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:35.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33400640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076d70 0x7fbb341a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:35.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb38a00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb34112760 0x7fbb3419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:35.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33400640 1 -- 172.21.15.3:0/1619001590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb34112760 msgr2=0x7fbb3419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:35.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33400640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb34112760 0x7fbb3419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33400640 1 -- 172.21.15.3:0/1619001590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34076450 msgr2=0x7fbb341a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:35.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33400640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34076450 0x7fbb341a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.447+0000 7fbb33400640 1 -- 172.21.15.3:0/1619001590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3419ed10 con 0x7fbb34076d70 2024-08-01T04:27:35.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.448+0000 7fbb33400640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076d70 0x7fbb341a4960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00e9f0 tx=0x7fbb1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:35.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.448+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00ccc0 con 0x7fbb34076d70 2024-08-01T04:27:35.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.448+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb1c005020 con 0x7fbb34076d70 2024-08-01T04:27:35.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.448+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0049e0 con 0x7fbb34076d70 2024-08-01T04:27:35.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.448+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3419f000 con 0x7fbb34076d70 2024-08-01T04:27:35.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.448+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb341ab2f0 con 0x7fbb34076d70 2024-08-01T04:27:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.450+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb34076830 con 0x7fbb34076d70 2024-08-01T04:27:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.452+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fbb1c005560 con 0x7fbb34076d70 2024-08-01T04:27:35.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.454+0000 7fbb30c00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbb180a2d70 0x7fbb180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:35.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.454+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7fbb1c0cf430 con 0x7fbb34076d70 2024-08-01T04:27:35.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.455+0000 7fbb33e00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbb180a2d70 0x7fbb180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:35.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.455+0000 7fbb33e00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbb180a2d70 0x7fbb180a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb240096c0 tx=0x7fbb24005a20 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:35.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.459+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbb1c0051d0 con 0x7fbb34076d70 2024-08-01T04:27:35.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-mon[37383]: pgmap v352: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-01T04:27:35.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-mon[37224]: pgmap v352: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-01T04:27:35.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.693+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbb3419fb10 con 0x7fbb34076d70 2024-08-01T04:27:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.698+0000 7fbb30c00640 1 -- 172.21.15.3:0/1619001590 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7fbb3419fb10 con 0x7fbb34076d70 2024-08-01T04:27:35.701 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-01T04:27:35.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.701+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbb180a2d70 msgr2=0x7fbb180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:35.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.701+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbb180a2d70 0x7fbb180a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbb240096c0 tx=0x7fbb24005a20 comp rx=0 tx=0).stop 2024-08-01T04:27:35.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.701+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076d70 msgr2=0x7fbb341a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:35.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.701+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076d70 0x7fbb341a4960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00e9f0 tx=0x7fbb1c00eeb0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 shutdown_connections 2024-08-01T04:27:35.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7fbb180a2d70 0x7fbb180a5230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb34112760 0x7fbb3419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb34076d70 0x7fbb341a4960 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 --2- 172.21.15.3:0/1619001590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb34076450 0x7fbb341a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:35.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 >> 172.21.15.3:0/1619001590 conn(0x7fbb340ff7a0 msgr2=0x7fbb34101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:35.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.702+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 shutdown_connections 2024-08-01T04:27:35.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:35.703+0000 7fbb3aa00640 1 -- 172.21.15.3:0/1619001590 wait complete. 2024-08-01T04:27:35.753 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: ignoring --setuser ceph since I am not root 2024-08-01T04:27:35.753 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: ignoring --setgroup ceph since I am not root 2024-08-01T04:27:35.753 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e[66665]: ignoring --setuser ceph since I am not root 2024-08-01T04:27:35.753 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e[66665]: ignoring --setgroup ceph since I am not root 2024-08-01T04:27:35.754 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-mds[66670]: -- 172.21.15.160:0/1884005591 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9975cb860 con 0x55e9975a8800 2024-08-01T04:27:35.754 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e[66665]: starting mds.e at 2024-08-01T04:27:36.046 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i[67845]: ignoring --setuser ceph since I am not root 2024-08-01T04:27:36.047 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i[67845]: ignoring --setgroup ceph since I am not root 2024-08-01T04:27:36.047 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-mds[67849]: -- 172.21.15.163:0/1999813204 <== mon.1 v2:172.21.15.163:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ce4dd3860 con 0x555ce4db1000 2024-08-01T04:27:36.047 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i[67845]: starting mds.i at 2024-08-01T04:27:36.047 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c[67542]: ignoring --setuser ceph since I am not root 2024-08-01T04:27:36.048 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c[67542]: ignoring --setgroup ceph since I am not root 2024-08-01T04:27:36.048 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-mds[67546]: -- 172.21.15.163:0/503771947 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f51d019860 con 0x55f51cff7000 2024-08-01T04:27:36.048 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:27:35 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c[67542]: starting mds.c at 2024-08-01T04:27:36.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:35 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a[32258]: 2024-08-01T04:27:35.695+0000 7fe1d3400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-01T04:27:36.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:27:35 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a[72667]: ignoring --setuser ceph since I am not root 2024-08-01T04:27:36.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:27:35 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a[72667]: ignoring --setgroup ceph since I am not root 2024-08-01T04:27:36.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:27:35 smithi003 ceph-mds[72671]: -- 172.21.15.3:0/2850174373 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aebca45860 con 0x55aebca22800 2024-08-01T04:27:36.050 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:27:35 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a[72667]: starting mds.a at 2024-08-01T04:27:36.152 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-mds[67286]: -- 172.21.15.160:0/3634639088 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ebb9d2f860 con 0x55ebb9d0d000 2024-08-01T04:27:36.153 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: starting mds.b at 2024-08-01T04:27:36.153 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h[66968]: ignoring --setuser ceph since I am not root 2024-08-01T04:27:36.153 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h[66968]: ignoring --setgroup ceph since I am not root 2024-08-01T04:27:36.153 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-mds[66972]: -- 172.21.15.160:0/2129048899 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582742c9860 con 0x5582742a7000 2024-08-01T04:27:36.153 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:27:35 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h[66968]: starting mds.h at 2024-08-01T04:27:36.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:36 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1619001590' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-01T04:27:36.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:36 smithi003 ceph-mon[32262]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-01T04:27:36.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:36 smithi003 ceph-mon[32262]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-01T04:27:36.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:36 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/1619001590' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-01T04:27:36.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:36 smithi003 ceph-mon[32262]: osdmap e102: 12 total, 12 up, 12 in 2024-08-01T04:27:36.480 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:36 smithi003 ceph-mon[32262]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-01T04:27:36.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-01T04:27:36.545 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:36 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1619001590' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-01T04:27:36.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:36 smithi163 ceph-mon[37383]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-01T04:27:36.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:36 smithi163 ceph-mon[37383]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-01T04:27:36.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:36 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/1619001590' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-01T04:27:36.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:36 smithi163 ceph-mon[37383]: osdmap e102: 12 total, 12 up, 12 in 2024-08-01T04:27:36.546 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:36 smithi163 ceph-mon[37383]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-01T04:27:36.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:36 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1619001590' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-01T04:27:36.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:36 smithi160 ceph-mon[37224]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-01T04:27:36.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:36 smithi160 ceph-mon[37224]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-01T04:27:36.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:36 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/1619001590' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-01T04:27:36.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:36 smithi160 ceph-mon[37224]: osdmap e102: 12 total, 12 up, 12 in 2024-08-01T04:27:36.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:36 smithi160 ceph-mon[37224]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-01T04:27:37.034 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:37.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 -- 172.21.15.3:0/2766400608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 msgr2=0x7f9828101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:37.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 --2- 172.21.15.3:0/2766400608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828101b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9810009890 tx=0x7f981002a360 comp rx=0 tx=0).stop 2024-08-01T04:27:37.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 -- 172.21.15.3:0/2766400608 shutdown_connections 2024-08-01T04:27:37.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 --2- 172.21.15.3:0/2766400608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9828102090 0x7f982810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:37.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 --2- 172.21.15.3:0/2766400608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828101b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 --2- 172.21.15.3:0/2766400608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98281076f0 0x7f9828107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.745+0000 7f9827e00640 1 -- 172.21.15.3:0/2766400608 >> 172.21.15.3:0/2766400608 conn(0x7f98280fd470 msgr2=0x7f98280ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.746+0000 7f9827e00640 1 -- 172.21.15.3:0/2766400608 shutdown_connections 2024-08-01T04:27:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.746+0000 7f9827e00640 1 -- 172.21.15.3:0/2766400608 wait complete. 2024-08-01T04:27:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 Processor -- start 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 -- start start 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9828102090 0x7f9828074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98281076f0 0x7f9828074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111ab0 con 0x7f98281016f0 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111930 con 0x7f98281076f0 2024-08-01T04:27:37.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.747+0000 7f9827e00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111c30 con 0x7f9828102090 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9826a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9827400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98281076f0 0x7f9828074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9826a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54020/0 (socket says 172.21.15.3:54020) 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9826a00640 1 -- 172.21.15.3:0/3138675878 learned_addr learned my addr 172.21.15.3:0/3138675878 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9826000640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9828102090 0x7f9828074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9827400640 1 -- 172.21.15.3:0/3138675878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9828102090 msgr2=0x7f9828074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9827400640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9828102090 0x7f9828074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:37.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9827400640 1 -- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 msgr2=0x7f9828074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9827400640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.748+0000 7f9827400640 1 -- 172.21.15.3:0/3138675878 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98281a64d0 con 0x7f98281076f0 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.749+0000 7f9827400640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98281076f0 0x7f9828074ca0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f982000ef00 tx=0x7f982000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.749+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820004b20 con 0x7f98281076f0 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.749+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98281a67c0 con 0x7f98281076f0 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.749+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9820003e50 con 0x7f98281076f0 2024-08-01T04:27:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.749+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982000b570 con 0x7f98281076f0 2024-08-01T04:27:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.749+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98281a6cd0 con 0x7f98281076f0 2024-08-01T04:27:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.750+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9828101b50 con 0x7f98281076f0 2024-08-01T04:27:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.752+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f982001d020 con 0x7f98281076f0 2024-08-01T04:27:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.753+0000 7f981f400640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f980c0a2d70 0x7f980c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.754+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8843+0+0 (secure 0 0 0) 0x7f98200cf410 con 0x7f98281076f0 2024-08-01T04:27:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.754+0000 7f9826a00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f980c0a2d70 0x7f980c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.754+0000 7f9826a00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f980c0a2d70 0x7f980c0a5230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9818009c10 tx=0x7f9818005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.758+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f982009be90 con 0x7f98281076f0 2024-08-01T04:27:37.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:37.991+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9828109c80 con 0x7f98281076f0 2024-08-01T04:27:37.994 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: pgmap v354: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-08-01T04:27:37.994 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6834/56531408,v1:172.21.15.160:6835/56531408] up:boot 2024-08-01T04:27:37.994 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6836/656581325,v1:172.21.15.160:6838/656581325] up:boot 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.160:6837/3740535418,v1:172.21.15.160:6839/3740535418] up:boot 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.163:6836/1224385407,v1:172.21.15.163:6838/1224385407] up:boot 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.163:6837/1801428210,v1:172.21.15.163:6839/1801428210] up:boot 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: mds.? [v2:172.21.15.3:6840/2434075140,v1:172.21.15.3:6841/2434075140] up:boot 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:27:37.995 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-01T04:27:37.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:27:37.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:27:37.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:37 smithi003 ceph-mon[32262]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:27:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: pgmap v354: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-08-01T04:27:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6834/56531408,v1:172.21.15.160:6835/56531408] up:boot 2024-08-01T04:27:38.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6836/656581325,v1:172.21.15.160:6838/656581325] up:boot 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.160:6837/3740535418,v1:172.21.15.160:6839/3740535418] up:boot 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.163:6836/1224385407,v1:172.21.15.163:6838/1224385407] up:boot 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.163:6837/1801428210,v1:172.21.15.163:6839/1801428210] up:boot 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: mds.? [v2:172.21.15.3:6840/2434075140,v1:172.21.15.3:6841/2434075140] up:boot 2024-08-01T04:27:38.047 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:27:38.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-01T04:27:38.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:27:38.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:27:38.048 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:37 smithi163 ceph-mon[37383]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:27:38.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: pgmap v354: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-08-01T04:27:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6834/56531408,v1:172.21.15.160:6835/56531408] up:boot 2024-08-01T04:27:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-01T04:27:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6836/656581325,v1:172.21.15.160:6838/656581325] up:boot 2024-08-01T04:27:38.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.160:6837/3740535418,v1:172.21.15.160:6839/3740535418] up:boot 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.163:6836/1224385407,v1:172.21.15.163:6838/1224385407] up:boot 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.163:6837/1801428210,v1:172.21.15.163:6839/1801428210] up:boot 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: mds.? [v2:172.21.15.3:6840/2434075140,v1:172.21.15.3:6841/2434075140] up:boot 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-01T04:27:38.154 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-01T04:27:38.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-01T04:27:38.155 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:37 smithi160 ceph-mon[37224]: from='mgr.14928 172.21.15.3:0/128331579' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-01T04:27:38.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.728+0000 7f981f400640 1 -- 172.21.15.3:0/3138675878 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f982009b790 con 0x7f98281076f0 2024-08-01T04:27:38.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f980c0a2d70 msgr2=0x7f980c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:38.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f980c0a2d70 0x7f980c0a5230 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9818009c10 tx=0x7f9818005f90 comp rx=0 tx=0).stop 2024-08-01T04:27:38.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98281076f0 msgr2=0x7f9828074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:38.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98281076f0 0x7f9828074ca0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f982000ef00 tx=0x7f982000c740 comp rx=0 tx=0).stop 2024-08-01T04:27:38.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 shutdown_connections 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98281076f0 0x7f9828074ca0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f980c0a2d70 0x7f980c0a5230 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9828102090 0x7f9828074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.731+0000 7f9827e00640 1 --2- 172.21.15.3:0/3138675878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98281016f0 0x7f9828074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.732+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 >> 172.21.15.3:0/3138675878 conn(0x7f98280fd470 msgr2=0x7f98280fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.732+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 shutdown_connections 2024-08-01T04:27:38.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:38.732+0000 7f9827e00640 1 -- 172.21.15.3:0/3138675878 wait complete. 2024-08-01T04:27:38.956 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:38 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/3138675878' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:27:38.956 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:38 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:27:39.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:38 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/3138675878' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:27:39.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:38 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:27:39.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:38 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/3138675878' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:27:39.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:38 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-01T04:27:40.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:39 smithi163 ceph-mon[37383]: pgmap v355: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 409 B/s wr, 5 op/s 2024-08-01T04:27:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:39 smithi163 ceph-mon[37383]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-01T04:27:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:39 smithi163 ceph-mon[37383]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-01T04:27:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:39 smithi163 ceph-mon[37383]: Cluster is now healthy 2024-08-01T04:27:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:39 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-01T04:27:40.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:39 smithi163 ceph-mon[37383]: fsmap 12 up:standby 2024-08-01T04:27:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:39 smithi003 ceph-mon[32262]: pgmap v355: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 409 B/s wr, 5 op/s 2024-08-01T04:27:40.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:39 smithi003 ceph-mon[32262]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-01T04:27:40.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:39 smithi003 ceph-mon[32262]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-01T04:27:40.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:39 smithi003 ceph-mon[32262]: Cluster is now healthy 2024-08-01T04:27:40.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:39 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-01T04:27:40.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:39 smithi003 ceph-mon[32262]: fsmap 12 up:standby 2024-08-01T04:27:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:39 smithi160 ceph-mon[37224]: pgmap v355: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.6 KiB/s rd, 409 B/s wr, 5 op/s 2024-08-01T04:27:40.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:39 smithi160 ceph-mon[37224]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-01T04:27:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:39 smithi160 ceph-mon[37224]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-01T04:27:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:39 smithi160 ceph-mon[37224]: Cluster is now healthy 2024-08-01T04:27:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:39 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-01T04:27:40.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:39 smithi160 ceph-mon[37224]: fsmap 12 up:standby 2024-08-01T04:27:41.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-01T04:27:42.006 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:42.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:41 smithi163 ceph-mon[37383]: pgmap v356: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 5 op/s 2024-08-01T04:27:42.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:41 smithi003 ceph-mon[32262]: pgmap v356: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 5 op/s 2024-08-01T04:27:42.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:41 smithi160 ceph-mon[37224]: pgmap v356: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 204 B/s wr, 5 op/s 2024-08-01T04:27:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 -- 172.21.15.3:0/567023011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec1086b0 msgr2=0x7f56ec114bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 --2- 172.21.15.3:0/567023011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec1086b0 0x7f56ec114bd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009930 tx=0x7f56e402a420 comp rx=0 tx=0).stop 2024-08-01T04:27:42.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 -- 172.21.15.3:0/567023011 shutdown_connections 2024-08-01T04:27:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 --2- 172.21.15.3:0/567023011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec1086b0 0x7f56ec114bd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 --2- 172.21.15.3:0/567023011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec107d50 0x7f56ec108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 --2- 172.21.15.3:0/567023011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec10dd90 0x7f56ec10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 -- 172.21.15.3:0/567023011 >> 172.21.15.3:0/567023011 conn(0x7f56ec0776b0 msgr2=0x7f56ec079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.697+0000 7f56f1a00640 1 -- 172.21.15.3:0/567023011 shutdown_connections 2024-08-01T04:27:42.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.698+0000 7f56f1a00640 1 -- 172.21.15.3:0/567023011 wait complete. 2024-08-01T04:27:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 Processor -- start 2024-08-01T04:27:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 -- start start 2024-08-01T04:27:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec107d50 0x7f56ec19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 0x7f56ec19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec10dd90 0x7f56ec1a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:42.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1180a0 con 0x7f56ec10dd90 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec117f20 con 0x7f56ec1086b0 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.699+0000 7f56f1a00640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec118220 con 0x7f56ec107d50 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 0x7f56ec19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56f1000640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec10dd90 0x7f56ec1a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56ebe00640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec107d50 0x7f56ec19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 0x7f56ec19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.3:60730/0 (socket says 172.21.15.3:60730) 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 -- 172.21.15.3:0/350141957 learned_addr learned my addr 172.21.15.3:0/350141957 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:42.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 -- 172.21.15.3:0/350141957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec107d50 msgr2=0x7f56ec19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec107d50 0x7f56ec19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 -- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec10dd90 msgr2=0x7f56ec1a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec10dd90 0x7f56ec1a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.700+0000 7f56eb400640 1 -- 172.21.15.3:0/350141957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec1a3500 con 0x7f56ec1086b0 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.701+0000 7f56eb400640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 0x7f56ec19ea90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f56d400e9f0 tx=0x7f56d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.701+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400cd80 con 0x7f56ec1086b0 2024-08-01T04:27:42.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.701+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec1a37f0 con 0x7f56ec1086b0 2024-08-01T04:27:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.701+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1ab0d0 con 0x7f56ec1086b0 2024-08-01T04:27:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.701+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d4010910 con 0x7f56ec1086b0 2024-08-01T04:27:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.701+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4018980 con 0x7f56ec1086b0 2024-08-01T04:27:42.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.702+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec108170 con 0x7f56ec1086b0 2024-08-01T04:27:42.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.705+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f56d4010ac0 con 0x7f56ec1086b0 2024-08-01T04:27:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.707+0000 7f56e8c00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f56d00a2d70 0x7f56d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.707+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8843+0+0 (secure 0 0 0) 0x7f56d40cfaf0 con 0x7f56ec1086b0 2024-08-01T04:27:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.707+0000 7f56ebe00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f56d00a2d70 0x7f56d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:42.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.708+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f56d409bf20 con 0x7f56ec1086b0 2024-08-01T04:27:42.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.708+0000 7f56ebe00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f56d00a2d70 0x7f56d00a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0099a0 tx=0x7f56dc009500 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:42.905+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f56ec110280 con 0x7f56ec1086b0 2024-08-01T04:27:43.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.732+0000 7f56e8c00640 1 -- 172.21.15.3:0/350141957 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f56d40a0220 con 0x7f56ec1086b0 2024-08-01T04:27:43.738 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-08-01T04:27:43.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f56d00a2d70 msgr2=0x7f56d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f56d00a2d70 0x7f56d00a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0099a0 tx=0x7f56dc009500 comp rx=0 tx=0).stop 2024-08-01T04:27:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 msgr2=0x7f56ec19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 0x7f56ec19ea90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f56d400e9f0 tx=0x7f56d400eeb0 comp rx=0 tx=0).stop 2024-08-01T04:27:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 shutdown_connections 2024-08-01T04:27:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f56ec10dd90 0x7f56ec1a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:43.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56ec1086b0 0x7f56ec19ea90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f56d00a2d70 0x7f56d00a5230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 --2- 172.21.15.3:0/350141957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56ec107d50 0x7f56ec19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.738+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 >> 172.21.15.3:0/350141957 conn(0x7f56ec0776b0 msgr2=0x7f56ec078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.739+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 shutdown_connections 2024-08-01T04:27:43.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:43.739+0000 7f56f1a00640 1 -- 172.21.15.3:0/350141957 wait complete. 2024-08-01T04:27:44.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:43 smithi163 ceph-mon[37383]: pgmap v357: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-08-01T04:27:44.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:43 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/350141957' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:44.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:43 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:44.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:43 smithi003 ceph-mon[32262]: pgmap v357: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-08-01T04:27:44.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:43 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/350141957' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:44.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:43 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:44.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:43 smithi160 ceph-mon[37224]: pgmap v357: 97 pgs: 97 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-08-01T04:27:44.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:43 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/350141957' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:44.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:43 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:44.728 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776 shell --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-01T04:27:45.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:44 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-01T04:27:45.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:44 smithi163 ceph-mon[37383]: osdmap e103: 12 total, 12 up, 12 in 2024-08-01T04:27:45.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:44 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-01T04:27:45.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:44 smithi003 ceph-mon[32262]: osdmap e103: 12 total, 12 up, 12 in 2024-08-01T04:27:45.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:44 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-01T04:27:45.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:44 smithi160 ceph-mon[37224]: osdmap e103: 12 total, 12 up, 12 in 2024-08-01T04:27:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/mon.a/config 2024-08-01T04:27:45.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 -- 172.21.15.3:0/245269355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 msgr2=0x7f6bc4114bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 --2- 172.21.15.3:0/245269355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc4114bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009be0 tx=0x7f6bbc02a6c0 comp rx=0 tx=0).stop 2024-08-01T04:27:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 -- 172.21.15.3:0/245269355 shutdown_connections 2024-08-01T04:27:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 --2- 172.21.15.3:0/245269355 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc4114bd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 --2- 172.21.15.3:0/245269355 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bc4107d50 0x7f6bc4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:45.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 --2- 172.21.15.3:0/245269355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc410dd90 0x7f6bc410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.865+0000 7f6bc3200640 1 -- 172.21.15.3:0/245269355 >> 172.21.15.3:0/245269355 conn(0x7f6bc40776b0 msgr2=0x7f6bc4079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.866+0000 7f6bc3200640 1 -- 172.21.15.3:0/245269355 shutdown_connections 2024-08-01T04:27:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.866+0000 7f6bc3200640 1 -- 172.21.15.3:0/245269355 wait complete. 2024-08-01T04:27:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.867+0000 7f6bc3200640 1 Processor -- start 2024-08-01T04:27:45.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 -- start start 2024-08-01T04:27:45.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 0x7f6bc419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:45.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bc410dd90 0x7f6bc41a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41180a0 con 0x7f6bc4107d50 2024-08-01T04:27:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4117f20 con 0x7f6bc41086b0 2024-08-01T04:27:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.868+0000 7f6bc3200640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4118220 con 0x7f6bc410dd90 2024-08-01T04:27:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 0x7f6bc419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:45.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:45.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1e00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 0x7f6bc419e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39884/0 (socket says 172.21.15.3:39884) 2024-08-01T04:27:45.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1e00640 1 -- 172.21.15.3:0/2991191016 learned_addr learned my addr 172.21.15.3:0/2991191016 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-08-01T04:27:45.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc2800640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bc410dd90 0x7f6bc41a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1400640 1 -- 172.21.15.3:0/2991191016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bc410dd90 msgr2=0x7f6bc41a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1400640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bc410dd90 0x7f6bc41a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:45.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1400640 1 -- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 msgr2=0x7f6bc419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1400640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 0x7f6bc419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1400640 1 -- 172.21.15.3:0/2991191016 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc41a34b0 con 0x7f6bc41086b0 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.869+0000 7f6bc1e00640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 0x7f6bc419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.870+0000 7f6bc1400640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc419ea40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.870+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000cd00 con 0x7f6bc41086b0 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.870+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb00035f0 con 0x7f6bc41086b0 2024-08-01T04:27:45.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.870+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc41a37a0 con 0x7f6bc41086b0 2024-08-01T04:27:45.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.870+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0010690 con 0x7f6bc41086b0 2024-08-01T04:27:45.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.870+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41ab080 con 0x7f6bc41086b0 2024-08-01T04:27:45.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.871+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc4108170 con 0x7f6bc41086b0 2024-08-01T04:27:45.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.875+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6bb00037a0 con 0x7f6bc41086b0 2024-08-01T04:27:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.877+0000 7f6bbaa00640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6ba80a2e40 0x7f6ba80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-01T04:27:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.877+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8289+0+0 (secure 0 0 0) 0x7f6bb00cf4b0 con 0x7f6bc41086b0 2024-08-01T04:27:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.877+0000 7f6bc1e00640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6ba80a2e40 0x7f6ba80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-01T04:27:45.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.878+0000 7f6bc1e00640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6ba80a2e40 0x7f6ba80a5300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4011440 tx=0x7f6bb4005f90 comp rx=0 tx=0).ready entity=mgr.14928 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-01T04:27:45.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:45.881+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6bb0009510 con 0x7f6bc41086b0 2024-08-01T04:27:46.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:45 smithi163 ceph-mon[37383]: pgmap v359: 65 pgs: 65 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2024-08-01T04:27:46.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:45 smithi003 ceph-mon[32262]: pgmap v359: 65 pgs: 65 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2024-08-01T04:27:46.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.080+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6bc402d5e0 con 0x7f6bc41086b0 2024-08-01T04:27:46.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:45 smithi160 ceph-mon[37224]: pgmap v359: 65 pgs: 65 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2024-08-01T04:27:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.773+0000 7f6bbaa00640 1 -- 172.21.15.3:0/2991191016 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f6bb009c150 con 0x7f6bc41086b0 2024-08-01T04:27:46.776 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-08-01T04:27:46.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.779+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6ba80a2e40 msgr2=0x7f6ba80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:46.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.779+0000 7f6bc3200640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6ba80a2e40 0x7f6ba80a5300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4011440 tx=0x7f6bb4005f90 comp rx=0 tx=0).stop 2024-08-01T04:27:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.779+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 msgr2=0x7f6bc419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-01T04:27:46.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.779+0000 7f6bc3200640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc419ea40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0005fd0 tx=0x7f6bb000ee50 comp rx=0 tx=0).stop 2024-08-01T04:27:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.780+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 shutdown_connections 2024-08-01T04:27:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.780+0000 7f6bc3200640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:6800/1464189073,v1:172.21.15.3:6801/1464189073] conn(0x7f6ba80a2e40 0x7f6ba80a5300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.780+0000 7f6bc3200640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bc410dd90 0x7f6bc41a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.780+0000 7f6bc3200640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bc41086b0 0x7f6bc419ea40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.780+0000 7f6bc3200640 1 --2- 172.21.15.3:0/2991191016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4107d50 0x7f6bc419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-01T04:27:46.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.780+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 >> 172.21.15.3:0/2991191016 conn(0x7f6bc40776b0 msgr2=0x7f6bc4078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-01T04:27:46.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.782+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 shutdown_connections 2024-08-01T04:27:46.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-08-01T04:27:46.782+0000 7f6bc3200640 1 -- 172.21.15.3:0/2991191016 wait complete. 2024-08-01T04:27:47.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:46 smithi163 ceph-mon[37383]: from='client.? 172.21.15.3:0/2991191016' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:47.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:46 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:47.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:46 smithi003 ceph-mon[32262]: from='client.? 172.21.15.3:0/2991191016' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:47.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:46 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:47.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:46 smithi160 ceph-mon[37224]: from='client.? 172.21.15.3:0/2991191016' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:47.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:46 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-01T04:27:47.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:27:47.743 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:27:47.813 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:27:47.871 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-01T04:27:47.871 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-01T04:27:47.944 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-01T04:27:47.974 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-01T04:27:47.998 INFO:tasks.cephadm:Stopping all daemons... 2024-08-01T04:27:47.999 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-01T04:27:47.999 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a 2024-08-01T04:27:48.045 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:47 smithi163 ceph-mon[37383]: pgmap v360: 65 pgs: 65 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2024-08-01T04:27:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:47 smithi163 ceph-mon[37383]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-01T04:27:48.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:27:47 smithi163 ceph-mon[37383]: osdmap e104: 12 total, 12 up, 12 in 2024-08-01T04:27:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:47 smithi003 ceph-mon[32262]: pgmap v360: 65 pgs: 65 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2024-08-01T04:27:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:47 smithi003 ceph-mon[32262]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-01T04:27:48.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:47 smithi003 ceph-mon[32262]: osdmap e104: 12 total, 12 up, 12 in 2024-08-01T04:27:48.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:47 smithi160 ceph-mon[37224]: pgmap v360: 65 pgs: 65 active+clean; 18 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail 2024-08-01T04:27:48.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:47 smithi160 ceph-mon[37224]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-01T04:27:48.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:47 smithi160 ceph-mon[37224]: osdmap e104: 12 total, 12 up, 12 in 2024-08-01T04:27:48.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:48 smithi003 systemd[1]: Stopping Ceph mon.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:27:49.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:48 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a[32258]: 2024-08-01T04:27:48.744+0000 7fe1da200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:27:49.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:48 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a[32258]: 2024-08-01T04:27:48.744+0000 7fe1da200640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-01T04:27:49.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:48 smithi003 podman[105039]: 2024-08-01 04:27:48.819786575 +0000 UTC m=+0.241152374 container died d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:27:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:49 smithi003 podman[105039]: 2024-08-01 04:27:49.299302134 +0000 UTC m=+0.720667927 container cleanup d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-01T04:27:49.798 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:49 smithi003 bash[105039]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a 2024-08-01T04:27:49.799 INFO:journalctl@ceph.mon.a.smithi003.stdout:Aug 01 04:27:49 smithi003 podman[105050]: 2024-08-01 04:27:49.543698286 +0000 UTC m=+0.709942528 container remove d8af0539c3effd04d22a775aa3b8172d77067e4fd15b399fdb63e416b3b2d5a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:27:50.099 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.a.service' 2024-08-01T04:27:50.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:27:50.389 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-01T04:27:50.389 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-01T04:27:50.390 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.b 2024-08-01T04:27:50.902 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:50 smithi160 systemd[1]: Stopping Ceph mon.b for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:27:52.652 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:52 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-b[37220]: 2024-08-01T04:27:52.285+0000 7f9b7e9c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:27:52.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:52 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-b[37220]: 2024-08-01T04:27:52.285+0000 7f9b7e9c9640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-01T04:27:52.653 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:52 smithi160 podman[72425]: 2024-08-01 04:27:52.510711968 +0000 UTC m=+1.087406942 container died 2ec6feaf7ed1159aa8350a27e30447626b43ef9a0012f750a16b885dc8040047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-01T04:27:57.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:57 smithi160 podman[72425]: 2024-08-01 04:27:57.141815467 +0000 UTC m=+5.718510433 container cleanup 2ec6feaf7ed1159aa8350a27e30447626b43ef9a0012f750a16b885dc8040047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS) 2024-08-01T04:27:57.402 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:27:57 smithi160 bash[72425]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-b 2024-08-01T04:28:00.298 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:28:00.030+0000 7fefd4600640 -1 client.14940: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-08-01T04:28:01.152 INFO:journalctl@ceph.mon.b.smithi160.stdout:Aug 01 04:28:00 smithi160 podman[72502]: 2024-08-01 04:28:00.669349797 +0000 UTC m=+8.149302087 container remove 2ec6feaf7ed1159aa8350a27e30447626b43ef9a0012f750a16b885dc8040047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-f9911e1, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-08-01T04:28:01.470 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.b.service' 2024-08-01T04:28:01.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:01.723 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-01T04:28:01.723 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-01T04:28:01.723 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.c 2024-08-01T04:28:02.046 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:28:01 smithi163 systemd[1]: Stopping Ceph mon.c for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:05.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:28:05 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-c[37379]: 2024-08-01T04:28:05.484+0000 7f5c55b9e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:28:05.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:28:05 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-c[37379]: 2024-08-01T04:28:05.484+0000 7f5c55b9e640 -1 mon.c@1(probing) e3 *** Got Signal Terminated *** 2024-08-01T04:28:05.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:28:05 smithi163 podman[73391]: 2024-08-01 04:28:05.627588375 +0000 UTC m=+2.410365458 container died 56cb51c00c8f82bb15165b7ca2caccf9027052faf2e3bb81b574c2dbf835fb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-c, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724) 2024-08-01T04:28:11.796 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:28:11 smithi163 podman[73391]: 2024-08-01 04:28:11.496863839 +0000 UTC m=+8.279640912 container cleanup 56cb51c00c8f82bb15165b7ca2caccf9027052faf2e3bb81b574c2dbf835fb07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-c, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=) 2024-08-01T04:28:11.797 INFO:journalctl@ceph.mon.c.smithi163.stdout:Aug 01 04:28:11 smithi163 bash[73391]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mon-c 2024-08-01T04:28:14.609 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mon.c.service' 2024-08-01T04:28:14.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:14.755 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-01T04:28:14.755 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-01T04:28:14.755 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x 2024-08-01T04:28:15.548 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:15 smithi003 systemd[1]: Stopping Ceph mgr.x for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:15.846 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:15 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x[32487]: 2024-08-01T04:28:15.739+0000 7ff0ac200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:28:16.188 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:15 smithi003 podman[105233]: 2024-08-01 04:28:15.852876185 +0000 UTC m=+0.283986596 container died ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1) 2024-08-01T04:28:16.798 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:16 smithi003 podman[105233]: 2024-08-01 04:28:16.462897384 +0000 UTC m=+0.894007834 container cleanup ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1) 2024-08-01T04:28:16.799 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:16 smithi003 bash[105233]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x 2024-08-01T04:28:16.799 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Aug 01 04:28:16 smithi003 podman[105245]: 2024-08-01 04:28:16.706565445 +0000 UTC m=+0.838544986 container remove ddbc642f976a63856010846f2b10158affadb3511cf43cdb56fb71ac659bfe3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f9911e17b12222fbe598fff73cd3ad10d20c5776, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-x, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-08-01T04:28:17.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.x.service' 2024-08-01T04:28:17.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:17.308 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-01T04:28:17.309 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-01T04:28:17.309 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.y 2024-08-01T04:28:17.652 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:28:17 smithi160 systemd[1]: Stopping Ceph mgr.y for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:19.372 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:28:19 smithi160 podman[72627]: 2024-08-01 04:28:19.101262251 +0000 UTC m=+1.113083944 container died 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-01T04:28:24.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:28:24 smithi160 podman[72627]: 2024-08-01 04:28:24.034162524 +0000 UTC m=+6.045984217 container cleanup 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-01T04:28:24.403 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:28:24 smithi160 bash[72627]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y 2024-08-01T04:28:26.998 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:28:26 smithi160 podman[72641]: 2024-08-01 04:28:26.720443856 +0000 UTC m=+7.616921903 container remove 958a0df1a1803d86816fc06a5027da077d6e0d4ecd49347b5ace0ffa5edd8559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, io.buildah.version=1.36.0) 2024-08-01T04:28:26.998 INFO:journalctl@ceph.mgr.y.smithi160.stdout:Aug 01 04:28:26 smithi160 systemd[1]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-08-01T04:28:27.014 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.y.service' 2024-08-01T04:28:27.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:27.050 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-01T04:28:27.050 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-01T04:28:27.051 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.z 2024-08-01T04:28:27.546 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:28:27 smithi163 systemd[1]: Stopping Ceph mgr.z for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:28.795 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:28:28 smithi163 podman[73532]: 2024-08-01 04:28:28.51110718 +0000 UTC m=+1.263199341 container died 79730b59046b5f87901894965d5e7714bf5d27385ead55004aefdaabb42b45ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-01T04:28:34.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:28:34 smithi163 podman[73532]: 2024-08-01 04:28:34.457752135 +0000 UTC m=+7.209844296 container cleanup 79730b59046b5f87901894965d5e7714bf5d27385ead55004aefdaabb42b45ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:28:34.796 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:28:34 smithi163 bash[73532]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z 2024-08-01T04:28:37.955 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:28:37 smithi163 podman[73547]: 2024-08-01 04:28:37.694285366 +0000 UTC m=+9.183132532 container remove 79730b59046b5f87901894965d5e7714bf5d27385ead55004aefdaabb42b45ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mgr-z, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-01T04:28:37.956 INFO:journalctl@ceph.mgr.z.smithi163.stdout:Aug 01 04:28:37 smithi163 systemd[1]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-08-01T04:28:37.959 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mgr.z.service' 2024-08-01T04:28:37.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:37.997 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-01T04:28:37.997 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-01T04:28:37.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.0 2024-08-01T04:28:38.549 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:38 smithi003 systemd[1]: Stopping Ceph osd.0 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:39.048 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:38 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0[46099]: 2024-08-01T04:28:38.617+0000 7f7d8dc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:28:39.049 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:38 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0[46099]: 2024-08-01T04:28:38.617+0000 7f7d8dc00640 -1 osd.0 104 *** Got signal Terminated *** 2024-08-01T04:28:39.049 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:38 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0[46099]: 2024-08-01T04:28:38.617+0000 7f7d8dc00640 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:28:45.032 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:44 smithi003 podman[105363]: 2024-08-01 04:28:44.714441564 +0000 UTC m=+6.249343031 container died 8d0671a641068d7b49d09ad6ece244727781b1e2e11c506ab0dacef605459616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-01T04:28:45.282 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:45 smithi003 podman[105363]: 2024-08-01 04:28:45.258714168 +0000 UTC m=+6.793615579 container cleanup 8d0671a641068d7b49d09ad6ece244727781b1e2e11c506ab0dacef605459616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-01T04:28:45.286 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:45 smithi003 bash[105363]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0 2024-08-01T04:28:45.799 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:45 smithi003 podman[105381]: 2024-08-01 04:28:45.535083764 +0000 UTC m=+0.790692201 container remove 8d0671a641068d7b49d09ad6ece244727781b1e2e11c506ab0dacef605459616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0, org.label-schema.build-date=20240724, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-01T04:28:46.740 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:46 smithi003 podman[105471]: 2024-08-01 04:28:46.325017021 +0000 UTC m=+0.035013885 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:28:46.741 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:46 smithi003 podman[105471]: 2024-08-01 04:28:46.4361461 +0000 UTC m=+0.146143002 container create d16259d3ab3f5b18df4bc869a0eef5bf14ccd8d7a6e553db0e85e264df646c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1) 2024-08-01T04:28:46.741 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:46 smithi003 podman[105471]: 2024-08-01 04:28:46.739560046 +0000 UTC m=+0.449556902 container init d16259d3ab3f5b18df4bc869a0eef5bf14ccd8d7a6e553db0e85e264df646c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:28:47.048 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:46 smithi003 podman[105471]: 2024-08-01 04:28:46.75979468 +0000 UTC m=+0.469791574 container start d16259d3ab3f5b18df4bc869a0eef5bf14ccd8d7a6e553db0e85e264df646c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-01T04:28:47.048 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:46 smithi003 podman[105471]: 2024-08-01 04:28:46.863856887 +0000 UTC m=+0.573853740 container attach d16259d3ab3f5b18df4bc869a0eef5bf14ccd8d7a6e553db0e85e264df646c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724) 2024-08-01T04:28:47.549 INFO:journalctl@ceph.osd.0.smithi003.stdout:Aug 01 04:28:47 smithi003 podman[105504]: 2024-08-01 04:28:47.339329585 +0000 UTC m=+0.038891435 container died d16259d3ab3f5b18df4bc869a0eef5bf14ccd8d7a6e553db0e85e264df646c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-01T04:28:48.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.0.service' 2024-08-01T04:28:48.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:48.212 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-01T04:28:48.212 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-01T04:28:48.212 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.1 2024-08-01T04:28:48.565 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:48 smithi160 systemd[1]: Stopping Ceph osd.1 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:48.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:48 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1[41657]: 2024-08-01T04:28:48.563+0000 7f546028c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:28:48.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:48 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1[41657]: 2024-08-01T04:28:48.563+0000 7f546028c640 -1 osd.1 104 *** Got signal Terminated *** 2024-08-01T04:28:48.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:48 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1[41657]: 2024-08-01T04:28:48.563+0000 7f546028c640 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:28:54.598 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:54 smithi160 podman[72760]: 2024-08-01 04:28:54.271315842 +0000 UTC m=+5.829330068 container died 42e3ab4677aa43169a9dfc559d0ff66d291a25ce19cca3d9f91296f09a9cd75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724) 2024-08-01T04:28:54.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:54 smithi160 podman[72760]: 2024-08-01 04:28:54.597429993 +0000 UTC m=+6.155444221 container cleanup 42e3ab4677aa43169a9dfc559d0ff66d291a25ce19cca3d9f91296f09a9cd75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1, RELEASE=squid-f9911e1, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-01T04:28:54.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:54 smithi160 bash[72760]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1 2024-08-01T04:28:54.903 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:54 smithi160 podman[72790]: 2024-08-01 04:28:54.832377291 +0000 UTC m=+0.556917590 container remove 42e3ab4677aa43169a9dfc559d0ff66d291a25ce19cca3d9f91296f09a9cd75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-01T04:28:55.513 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:55 smithi160 podman[72878]: 2024-08-01 04:28:55.135468848 +0000 UTC m=+0.017164940 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:28:55.513 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:55 smithi160 podman[72878]: 2024-08-01 04:28:55.258103923 +0000 UTC m=+0.139800014 container create 6fd85361fa7f5e7f2bd9b948d99d17086f594ec3404729fbf5a07d33a964e3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240724, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-f9911e1) 2024-08-01T04:28:55.902 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:55 smithi160 podman[72878]: 2024-08-01 04:28:55.511258374 +0000 UTC m=+0.392954475 container init 6fd85361fa7f5e7f2bd9b948d99d17086f594ec3404729fbf5a07d33a964e3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:28:55.903 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:55 smithi160 podman[72878]: 2024-08-01 04:28:55.514649175 +0000 UTC m=+0.396345268 container start 6fd85361fa7f5e7f2bd9b948d99d17086f594ec3404729fbf5a07d33a964e3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:28:55.903 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:55 smithi160 podman[72878]: 2024-08-01 04:28:55.652773861 +0000 UTC m=+0.534469966 container attach 6fd85361fa7f5e7f2bd9b948d99d17086f594ec3404729fbf5a07d33a964e3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-01T04:28:56.341 INFO:journalctl@ceph.osd.1.smithi160.stdout:Aug 01 04:28:56 smithi160 podman[72878]: 2024-08-01 04:28:56.027177687 +0000 UTC m=+0.908873781 container died 6fd85361fa7f5e7f2bd9b948d99d17086f594ec3404729fbf5a07d33a964e3a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-01T04:28:56.663 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.1.service' 2024-08-01T04:28:56.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:28:56.705 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-01T04:28:56.705 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-01T04:28:56.705 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.2 2024-08-01T04:28:57.022 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:28:56 smithi163 systemd[1]: Stopping Ceph osd.2 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:28:57.295 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:28:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2[42253]: 2024-08-01T04:28:57.020+0000 7ff85edff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:28:57.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:28:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2[42253]: 2024-08-01T04:28:57.020+0000 7ff85edff640 -1 osd.2 104 *** Got signal Terminated *** 2024-08-01T04:28:57.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:28:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2[42253]: 2024-08-01T04:28:57.020+0000 7ff85edff640 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:02.998 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:02 smithi163 podman[73664]: 2024-08-01 04:29:02.695112305 +0000 UTC m=+5.791489314 container died 611b60314c91b4527db106cd412e1c1a2286bc931167bbd905ebc165ab6a8248 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:03.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 podman[73664]: 2024-08-01 04:29:03.105460093 +0000 UTC m=+6.201837101 container cleanup 611b60314c91b4527db106cd412e1c1a2286bc931167bbd905ebc165ab6a8248 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-01T04:29:03.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 bash[73664]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2 2024-08-01T04:29:03.678 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 podman[73679]: 2024-08-01 04:29:03.365543686 +0000 UTC m=+0.667210165 container remove 611b60314c91b4527db106cd412e1c1a2286bc931167bbd905ebc165ab6a8248 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:29:03.979 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 podman[73769]: 2024-08-01 04:29:03.580346823 +0000 UTC m=+0.017280314 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:03.979 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 podman[73769]: 2024-08-01 04:29:03.703763611 +0000 UTC m=+0.140697090 container create 1bf695a3912bac61a23da12085a4ce48e699ec8312a4e2d11a62abe58fb30bc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-01T04:29:04.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 podman[73769]: 2024-08-01 04:29:03.977887474 +0000 UTC m=+0.414820955 container init 1bf695a3912bac61a23da12085a4ce48e699ec8312a4e2d11a62abe58fb30bc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-08-01T04:29:04.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:03 smithi163 podman[73769]: 2024-08-01 04:29:03.981770577 +0000 UTC m=+0.418704059 container start 1bf695a3912bac61a23da12085a4ce48e699ec8312a4e2d11a62abe58fb30bc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:29:04.296 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:04 smithi163 podman[73769]: 2024-08-01 04:29:04.120055809 +0000 UTC m=+0.556989301 container attach 1bf695a3912bac61a23da12085a4ce48e699ec8312a4e2d11a62abe58fb30bc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:04.690 INFO:journalctl@ceph.osd.2.smithi163.stdout:Aug 01 04:29:04 smithi163 podman[73769]: 2024-08-01 04:29:04.414925854 +0000 UTC m=+0.851859346 container died 1bf695a3912bac61a23da12085a4ce48e699ec8312a4e2d11a62abe58fb30bc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD) 2024-08-01T04:29:05.034 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.2.service' 2024-08-01T04:29:05.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:05.073 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-01T04:29:05.073 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-01T04:29:05.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.3 2024-08-01T04:29:05.548 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:05 smithi003 systemd[1]: Stopping Ceph osd.3 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:06.049 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:05 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3[52658]: 2024-08-01T04:29:05.718+0000 7fdfc7000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:06.049 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:05 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3[52658]: 2024-08-01T04:29:05.718+0000 7fdfc7000640 -1 osd.3 104 *** Got signal Terminated *** 2024-08-01T04:29:06.049 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:05 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3[52658]: 2024-08-01T04:29:05.718+0000 7fdfc7000640 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:12.277 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:11 smithi003 podman[105584]: 2024-08-01 04:29:11.964038153 +0000 UTC m=+6.401448044 container died 4d0a0f1a7e69f4b2fa617c2c34de8cbb4e688a81cb33945f82ff24f60f378c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:12.798 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:12 smithi003 podman[105584]: 2024-08-01 04:29:12.544191892 +0000 UTC m=+6.981601818 container cleanup 4d0a0f1a7e69f4b2fa617c2c34de8cbb4e688a81cb33945f82ff24f60f378c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3, ceph=True, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-01T04:29:12.798 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:12 smithi003 bash[105584]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3 2024-08-01T04:29:13.048 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:12 smithi003 podman[105603]: 2024-08-01 04:29:12.796264015 +0000 UTC m=+0.802398008 container remove 4d0a0f1a7e69f4b2fa617c2c34de8cbb4e688a81cb33945f82ff24f60f378c2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724) 2024-08-01T04:29:14.003 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:13 smithi003 podman[105690]: 2024-08-01 04:29:13.541742148 +0000 UTC m=+0.038500257 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:14.003 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:13 smithi003 podman[105690]: 2024-08-01 04:29:13.663871923 +0000 UTC m=+0.160630043 container create b12bec360994ae746b438c52a3282ac1f4cd84ce862e42084227725d9bff1157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-01T04:29:14.003 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:14 smithi003 podman[105690]: 2024-08-01 04:29:14.001252543 +0000 UTC m=+0.498010724 container init b12bec360994ae746b438c52a3282ac1f4cd84ce862e42084227725d9bff1157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-01T04:29:14.298 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:14 smithi003 podman[105690]: 2024-08-01 04:29:14.013475288 +0000 UTC m=+0.510233389 container start b12bec360994ae746b438c52a3282ac1f4cd84ce862e42084227725d9bff1157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0) 2024-08-01T04:29:14.299 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:14 smithi003 podman[105690]: 2024-08-01 04:29:14.133555603 +0000 UTC m=+0.630313798 container attach b12bec360994ae746b438c52a3282ac1f4cd84ce862e42084227725d9bff1157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True) 2024-08-01T04:29:14.694 INFO:journalctl@ceph.osd.3.smithi003.stdout:Aug 01 04:29:14 smithi003 podman[105713]: 2024-08-01 04:29:14.384250744 +0000 UTC m=+0.037345560 container died b12bec360994ae746b438c52a3282ac1f4cd84ce862e42084227725d9bff1157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-f9911e1) 2024-08-01T04:29:15.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.3.service' 2024-08-01T04:29:15.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:15.248 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-01T04:29:15.248 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-01T04:29:15.248 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.4 2024-08-01T04:29:15.568 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:15 smithi160 systemd[1]: Stopping Ceph osd.4 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:15.902 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:15 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4[48258]: 2024-08-01T04:29:15.566+0000 7f6c76b1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:15.903 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:15 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4[48258]: 2024-08-01T04:29:15.566+0000 7f6c76b1a640 -1 osd.4 104 *** Got signal Terminated *** 2024-08-01T04:29:15.903 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:15 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4[48258]: 2024-08-01T04:29:15.566+0000 7f6c76b1a640 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:21.477 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:21 smithi160 podman[72983]: 2024-08-01 04:29:21.188093307 +0000 UTC m=+5.743007602 container died d8b75ceafc429ab41f25823031b7536f65bab4e83173a626ef65386b4ccb1f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-01T04:29:21.827 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:21 smithi160 podman[72983]: 2024-08-01 04:29:21.583934327 +0000 UTC m=+6.138848611 container cleanup d8b75ceafc429ab41f25823031b7536f65bab4e83173a626ef65386b4ccb1f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-01T04:29:21.828 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:21 smithi160 bash[72983]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4 2024-08-01T04:29:22.144 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:21 smithi160 podman[73000]: 2024-08-01 04:29:21.826960808 +0000 UTC m=+0.641316330 container remove d8b75ceafc429ab41f25823031b7536f65bab4e83173a626ef65386b4ccb1f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:29:22.402 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:22 smithi160 podman[73093]: 2024-08-01 04:29:22.045811507 +0000 UTC m=+0.017828862 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:22.403 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:22 smithi160 podman[73093]: 2024-08-01 04:29:22.152726091 +0000 UTC m=+0.124743451 container create c817f5f6eda0ca6ac34214cfaa797f3c242ef420b7d1b5c10107f36af99ea259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:29:22.903 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:22 smithi160 podman[73093]: 2024-08-01 04:29:22.430128641 +0000 UTC m=+0.402146008 container init c817f5f6eda0ca6ac34214cfaa797f3c242ef420b7d1b5c10107f36af99ea259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-01T04:29:22.903 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:22 smithi160 podman[73093]: 2024-08-01 04:29:22.433190151 +0000 UTC m=+0.405207508 container start c817f5f6eda0ca6ac34214cfaa797f3c242ef420b7d1b5c10107f36af99ea259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1) 2024-08-01T04:29:22.903 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:22 smithi160 podman[73093]: 2024-08-01 04:29:22.530144566 +0000 UTC m=+0.502161939 container attach c817f5f6eda0ca6ac34214cfaa797f3c242ef420b7d1b5c10107f36af99ea259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:29:22.903 INFO:journalctl@ceph.osd.4.smithi160.stdout:Aug 01 04:29:22 smithi160 podman[73093]: 2024-08-01 04:29:22.624399516 +0000 UTC m=+0.596416875 container died c817f5f6eda0ca6ac34214cfaa797f3c242ef420b7d1b5c10107f36af99ea259 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:23.298 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.4.service' 2024-08-01T04:29:23.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:23.337 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-01T04:29:23.338 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-01T04:29:23.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.5 2024-08-01T04:29:23.652 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:23 smithi163 systemd[1]: Stopping Ceph osd.5 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:24.045 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:23 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5[49143]: 2024-08-01T04:29:23.651+0000 7fa2d0fcb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:24.046 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:23 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5[49143]: 2024-08-01T04:29:23.651+0000 7fa2d0fcb640 -1 osd.5 104 *** Got signal Terminated *** 2024-08-01T04:29:24.046 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:23 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5[49143]: 2024-08-01T04:29:23.651+0000 7fa2d0fcb640 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:29.620 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:29 smithi163 podman[73885]: 2024-08-01 04:29:29.329296509 +0000 UTC m=+5.793561259 container died dfd831331f2ba0a8700ff933fcbbbd16c360147819a3da1871f086effc6c9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-01T04:29:29.962 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:29 smithi163 podman[73885]: 2024-08-01 04:29:29.718763542 +0000 UTC m=+6.183028358 container cleanup dfd831331f2ba0a8700ff933fcbbbd16c360147819a3da1871f086effc6c9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:29:29.962 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:29 smithi163 bash[73885]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5 2024-08-01T04:29:30.298 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:29 smithi163 podman[73900]: 2024-08-01 04:29:29.96178412 +0000 UTC m=+0.635292283 container remove dfd831331f2ba0a8700ff933fcbbbd16c360147819a3da1871f086effc6c9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724) 2024-08-01T04:29:30.658 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:30 smithi163 podman[73986]: 2024-08-01 04:29:30.282070238 +0000 UTC m=+0.017262363 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:30.658 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:30 smithi163 podman[73986]: 2024-08-01 04:29:30.412996123 +0000 UTC m=+0.148188235 container create 465ff0b017487992f61b6db3ec6242966e8c19bd635690e47e2cbea97d072d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:29:31.046 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:30 smithi163 podman[73986]: 2024-08-01 04:29:30.657459935 +0000 UTC m=+0.392652051 container init 465ff0b017487992f61b6db3ec6242966e8c19bd635690e47e2cbea97d072d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS) 2024-08-01T04:29:31.046 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:30 smithi163 podman[73986]: 2024-08-01 04:29:30.661501733 +0000 UTC m=+0.396693855 container start 465ff0b017487992f61b6db3ec6242966e8c19bd635690e47e2cbea97d072d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1) 2024-08-01T04:29:31.046 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:30 smithi163 podman[73986]: 2024-08-01 04:29:30.749333959 +0000 UTC m=+0.484526083 container attach 465ff0b017487992f61b6db3ec6242966e8c19bd635690e47e2cbea97d072d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:31.047 INFO:journalctl@ceph.osd.5.smithi163.stdout:Aug 01 04:29:30 smithi163 podman[73986]: 2024-08-01 04:29:30.848814774 +0000 UTC m=+0.584006897 container died 465ff0b017487992f61b6db3ec6242966e8c19bd635690e47e2cbea97d072d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-01T04:29:31.469 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.5.service' 2024-08-01T04:29:31.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:31.507 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-01T04:29:31.507 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-01T04:29:31.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.6 2024-08-01T04:29:32.049 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:31 smithi003 systemd[1]: Stopping Ceph osd.6 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:32.549 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:32 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6[59180]: 2024-08-01T04:29:32.132+0000 7f18e6200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:32.549 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:32 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6[59180]: 2024-08-01T04:29:32.132+0000 7f18e6200640 -1 osd.6 104 *** Got signal Terminated *** 2024-08-01T04:29:32.549 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:32 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6[59180]: 2024-08-01T04:29:32.132+0000 7f18e6200640 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:38.481 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:38 smithi003 podman[105806]: 2024-08-01 04:29:38.189300606 +0000 UTC m=+6.224894987 container died 5db8b66546d9acf01ee33b764bdce318359fd55e19b6120dd12702d20d7b655d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1) 2024-08-01T04:29:38.798 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:38 smithi003 podman[105806]: 2024-08-01 04:29:38.69761268 +0000 UTC m=+6.733207050 container cleanup 5db8b66546d9acf01ee33b764bdce318359fd55e19b6120dd12702d20d7b655d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:29:38.798 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:38 smithi003 bash[105806]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6 2024-08-01T04:29:39.298 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:38 smithi003 podman[105825]: 2024-08-01 04:29:38.924486605 +0000 UTC m=+0.721265381 container remove 5db8b66546d9acf01ee33b764bdce318359fd55e19b6120dd12702d20d7b655d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-01T04:29:40.048 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:39 smithi003 podman[105915]: 2024-08-01 04:29:39.696312589 +0000 UTC m=+0.041103935 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:40.049 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:39 smithi003 podman[105915]: 2024-08-01 04:29:39.800416959 +0000 UTC m=+0.145208327 container create 4e07214075465972b11eee5a5b8789d298a4955826ae71de8b4e150ac32018ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-01T04:29:40.429 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:40 smithi003 podman[105915]: 2024-08-01 04:29:40.095544624 +0000 UTC m=+0.440336005 container init 4e07214075465972b11eee5a5b8789d298a4955826ae71de8b4e150ac32018ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-01T04:29:40.429 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:40 smithi003 podman[105915]: 2024-08-01 04:29:40.111853384 +0000 UTC m=+0.456644752 container start 4e07214075465972b11eee5a5b8789d298a4955826ae71de8b4e150ac32018ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-01T04:29:40.429 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:40 smithi003 podman[105915]: 2024-08-01 04:29:40.202760024 +0000 UTC m=+0.547551452 container attach 4e07214075465972b11eee5a5b8789d298a4955826ae71de8b4e150ac32018ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-01T04:29:40.429 INFO:journalctl@ceph.osd.6.smithi003.stdout:Aug 01 04:29:40 smithi003 podman[105915]: 2024-08-01 04:29:40.42765383 +0000 UTC m=+0.772445201 container died 4e07214075465972b11eee5a5b8789d298a4955826ae71de8b4e150ac32018ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:41.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.6.service' 2024-08-01T04:29:41.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:41.371 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-01T04:29:41.371 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-01T04:29:41.371 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.7 2024-08-01T04:29:41.652 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:41 smithi160 systemd[1]: Stopping Ceph osd.7 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:41.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:41 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7[54715]: 2024-08-01T04:29:41.643+0000 7fdb9912c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:41.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:41 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7[54715]: 2024-08-01T04:29:41.643+0000 7fdb9912c640 -1 osd.7 104 *** Got signal Terminated *** 2024-08-01T04:29:41.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:41 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7[54715]: 2024-08-01T04:29:41.643+0000 7fdb9912c640 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:46.212 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:45 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:45.890+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:47.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:46 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:46.924+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:47.737 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:47 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:47.464+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:47.737 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:47 smithi160 podman[73196]: 2024-08-01 04:29:47.463475336 +0000 UTC m=+5.901533391 container died b0b1eb800012274ea73c001d41e864fbb29dc4d93f860a8402b862f3eed3ad81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1) 2024-08-01T04:29:48.086 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:47 smithi160 podman[73196]: 2024-08-01 04:29:47.834667857 +0000 UTC m=+6.272725914 container cleanup b0b1eb800012274ea73c001d41e864fbb29dc4d93f860a8402b862f3eed3ad81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:48.087 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:47 smithi160 bash[73196]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7 2024-08-01T04:29:48.247 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:47 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:47.837+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:48.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:47 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:47.882+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:48.395 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73214]: 2024-08-01 04:29:48.085706786 +0000 UTC m=+0.624006395 container remove b0b1eb800012274ea73c001d41e864fbb29dc4d93f860a8402b862f3eed3ad81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-01T04:29:48.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:48 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:48.488+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:48.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73299]: 2024-08-01 04:29:48.394728926 +0000 UTC m=+0.108993951 container create 6dca24f76d5fb5040d66c49b784b0942daa7ca30e3b404e92065381a2c188037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1) 2024-08-01T04:29:48.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73299]: 2024-08-01 04:29:48.303004133 +0000 UTC m=+0.017269168 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:48.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73299]: 2024-08-01 04:29:48.614110447 +0000 UTC m=+0.328375488 container init 6dca24f76d5fb5040d66c49b784b0942daa7ca30e3b404e92065381a2c188037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1) 2024-08-01T04:29:48.653 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73299]: 2024-08-01 04:29:48.617326247 +0000 UTC m=+0.331591273 container start 6dca24f76d5fb5040d66c49b784b0942daa7ca30e3b404e92065381a2c188037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-01T04:29:49.109 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73299]: 2024-08-01 04:29:48.697559969 +0000 UTC m=+0.411825020 container attach 6dca24f76d5fb5040d66c49b784b0942daa7ca30e3b404e92065381a2c188037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-01T04:29:49.109 INFO:journalctl@ceph.osd.7.smithi160.stdout:Aug 01 04:29:48 smithi160 podman[73299]: 2024-08-01 04:29:48.804451585 +0000 UTC m=+0.518716613 container died 6dca24f76d5fb5040d66c49b784b0942daa7ca30e3b404e92065381a2c188037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-7-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-08-01T04:29:49.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:48 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:48.916+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:49.298 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:48 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:48.871+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:49.428 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.7.service' 2024-08-01T04:29:49.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:49.466 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-01T04:29:49.466 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-01T04:29:49.467 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.8 2024-08-01T04:29:49.682 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:49 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:49.508+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:49.796 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:49 smithi163 systemd[1]: Stopping Ceph osd.8 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:50.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:49 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:49.885+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:50.296 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:49 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8[55910]: 2024-08-01T04:29:49.804+0000 7f79a9d72640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:50.296 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:49 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8[55910]: 2024-08-01T04:29:49.804+0000 7f79a9d72640 -1 osd.8 104 *** Got signal Terminated *** 2024-08-01T04:29:50.296 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:49 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8[55910]: 2024-08-01T04:29:49.804+0000 7f79a9d72640 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:50.298 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:49 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:49.876+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:50.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:50 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:50.469+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:51.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:50 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:50.897+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:51.298 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:50 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:50.833+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:51.804 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:51 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:51.421+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:52.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:51 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:51.919+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:52.298 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:51 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:51.808+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:52.903 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:52 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:52.461+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:53.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:52 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:52.778+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:53.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:52 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:52.943+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:53.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:53 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:53.462+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:54.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:53 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:53.753+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:54.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:53 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:53.896+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:54.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:54 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:54.463+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:55.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:54 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:54.781+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:55.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:54 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:54.859+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:55.683 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:55 smithi163 podman[74102]: 2024-08-01 04:29:55.40854248 +0000 UTC m=+5.744486354 container died 17f2a7b308af370cca35986ef02833bd659f5b9aa9aec8e245c806290bed00a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-01T04:29:55.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:55 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:55.449+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:55.991 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:55 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:55.910+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:55.992 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:55 smithi163 podman[74102]: 2024-08-01 04:29:55.772803174 +0000 UTC m=+6.108747036 container cleanup 17f2a7b308af370cca35986ef02833bd659f5b9aa9aec8e245c806290bed00a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:29:55.992 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:55 smithi163 bash[74102]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8 2024-08-01T04:29:56.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:55 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:55.760+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:56.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:55 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:55.760+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:29:56.295 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:55 smithi163 podman[74117]: 2024-08-01 04:29:55.990833788 +0000 UTC m=+0.584712082 container remove 17f2a7b308af370cca35986ef02833bd659f5b9aa9aec8e245c806290bed00a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-01T04:29:56.678 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:56 smithi163 podman[74206]: 2024-08-01 04:29:56.296809916 +0000 UTC m=+0.017177644 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:29:56.679 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:56 smithi163 podman[74206]: 2024-08-01 04:29:56.433706073 +0000 UTC m=+0.154073794 container create c5d61ef410b5690254281c46a07d1967d3f0eede1e102fedea58fb754a1364fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-08-01T04:29:56.679 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:56 smithi163 podman[74206]: 2024-08-01 04:29:56.677966835 +0000 UTC m=+0.398334564 container init c5d61ef410b5690254281c46a07d1967d3f0eede1e102fedea58fb754a1364fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1) 2024-08-01T04:29:56.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:56.406+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:56.903 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:56.406+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:29:57.046 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:56 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:56.890+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:57.046 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:56 smithi163 podman[74206]: 2024-08-01 04:29:56.681333605 +0000 UTC m=+0.401701318 container start c5d61ef410b5690254281c46a07d1967d3f0eede1e102fedea58fb754a1364fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-01T04:29:57.047 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:56 smithi163 podman[74206]: 2024-08-01 04:29:56.811552213 +0000 UTC m=+0.531919926 container attach c5d61ef410b5690254281c46a07d1967d3f0eede1e102fedea58fb754a1364fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-01T04:29:57.047 INFO:journalctl@ceph.osd.8.smithi163.stdout:Aug 01 04:29:56 smithi163 podman[74206]: 2024-08-01 04:29:56.872126963 +0000 UTC m=+0.592494676 container died c5d61ef410b5690254281c46a07d1967d3f0eede1e102fedea58fb754a1364fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-01T04:29:57.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:56 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:56.756+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:57.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:56 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:56.756+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:29:57.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.8.service' 2024-08-01T04:29:57.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:29:57.572 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-01T04:29:57.572 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-01T04:29:57.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.9 2024-08-01T04:29:57.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:57.390+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:57.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:57 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:57.390+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:29:58.045 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:57.933+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:58.046 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:57 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:57.933+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:29:58.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:57 smithi003 systemd[1]: Stopping Ceph osd.9 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:29:58.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:57 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:57.731+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:58.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:57 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:57.731+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:29:58.548 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:58.133+0000 7fc789a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:29:58.549 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:58.133+0000 7fc789a00640 -1 osd.9 104 *** Got signal Terminated *** 2024-08-01T04:29:58.549 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:58.133+0000 7fc789a00640 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:29:58.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:58.432+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:58.902 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:58 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:58.432+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:29:59.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:58.747+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:29:59.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:58 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:58.747+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:29:59.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:58 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:58.979+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:29:59.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:58 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:58.979+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:29:59.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:59.392+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:29:59.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:29:59 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:29:59.392+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:00.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:59 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:59.770+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:30:00.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:29:59 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:29:59.770+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:30:00.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:59 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:59.952+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:00.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:29:59 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:29:59.952+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:00.679 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:00 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:00.409+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:00.679 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:00 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:00.409+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:01.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:30:00.775+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:30:01.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:00 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:30:00.775+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:30:01.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:00 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:00.950+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:01.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:00 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:00.950+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:01.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:01 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:01.363+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:01.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:01 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:01.363+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:02.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:01 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:30:01.796+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:30:02.049 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:01 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:30:01.796+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:30:02.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:01 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:01.992+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:02.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:01 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:01.992+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:02.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:02 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:02.340+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:02.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:02 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:02.340+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:03.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:02 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:30:02.763+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.571317+0000 front 2024-08-01T04:28:43.570782+0000 (oldest deadline 2024-08-01T04:29:47.671025+0000) 2024-08-01T04:30:03.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:02 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9[65686]: 2024-08-01T04:30:02.763+0000 7fc780400640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:52.174882+0000 front 2024-08-01T04:28:52.174463+0000 (oldest deadline 2024-08-01T04:29:55.074415+0000) 2024-08-01T04:30:03.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:02 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:02.952+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:03.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:02 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:02.952+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:03.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:03 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:03.346+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:03.654 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:03 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:03.346+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:04.229 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:03 smithi003 podman[106029]: 2024-08-01 04:30:03.969852568 +0000 UTC m=+5.964877631 container died 69097757d78078298b38cbea40ae1e840f2a69cb4729482bf6e2e1eb4e8158f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1) 2024-08-01T04:30:04.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:03 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:03.910+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:04.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:03 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:03.910+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:04.548 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:04 smithi003 podman[106029]: 2024-08-01 04:30:04.400399026 +0000 UTC m=+6.395424048 container cleanup 69097757d78078298b38cbea40ae1e840f2a69cb4729482bf6e2e1eb4e8158f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:04.548 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:04 smithi003 bash[106029]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9 2024-08-01T04:30:04.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:04 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:04.366+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:04.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:04 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:04.366+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:04.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:04 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:04.366+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:05.048 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:04 smithi003 podman[106048]: 2024-08-01 04:30:04.64357265 +0000 UTC m=+0.657176338 container remove 69097757d78078298b38cbea40ae1e840f2a69cb4729482bf6e2e1eb4e8158f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-01T04:30:05.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:04 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:04.948+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:05.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:04 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:04.948+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:05.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:05 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:05.320+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:05.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:05 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:05.320+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:05.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:05 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:05.320+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:06.028 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:05 smithi003 podman[106139]: 2024-08-01 04:30:05.474920478 +0000 UTC m=+0.034365568 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:30:06.028 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:05 smithi003 podman[106139]: 2024-08-01 04:30:05.585949517 +0000 UTC m=+0.145394630 container create a5f555f163a0610364937249bdf99a27e7ece712fc016c52ab74186f86bed6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:06.028 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:06 smithi003 podman[106139]: 2024-08-01 04:30:06.025813423 +0000 UTC m=+0.585258540 container init a5f555f163a0610364937249bdf99a27e7ece712fc016c52ab74186f86bed6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, ceph=True, GIT_BRANCH=HEAD) 2024-08-01T04:30:06.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:05 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:05.992+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:06.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:05 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:05.992+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:06.298 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:06 smithi003 podman[106139]: 2024-08-01 04:30:06.037688243 +0000 UTC m=+0.597133360 container start a5f555f163a0610364937249bdf99a27e7ece712fc016c52ab74186f86bed6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-01T04:30:06.298 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:06 smithi003 podman[106139]: 2024-08-01 04:30:06.1218554 +0000 UTC m=+0.681300521 container attach a5f555f163a0610364937249bdf99a27e7ece712fc016c52ab74186f86bed6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:30:06.638 INFO:journalctl@ceph.osd.9.smithi003.stdout:Aug 01 04:30:06 smithi003 podman[106139]: 2024-08-01 04:30:06.343454771 +0000 UTC m=+0.902899891 container died a5f555f163a0610364937249bdf99a27e7ece712fc016c52ab74186f86bed6b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:30:06.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:06 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:06.323+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:06.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:06 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:06.323+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:06.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:06 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:06.323+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:06.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.9.service' 2024-08-01T04:30:07.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:07.091 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-01T04:30:07.092 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-01T04:30:07.092 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.10 2024-08-01T04:30:07.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:07 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:07.005+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:07.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:07 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:07.005+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:07.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:07 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:07.005+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:07.387 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 systemd[1]: Stopping Ceph osd.10 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:07.387 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:07.284+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:07.387 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:07.284+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:07.387 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:07.284+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:07.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:07.385+0000 7f7779de0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:07.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:07.385+0000 7f7779de0640 -1 osd.10 104 *** Got signal Terminated *** 2024-08-01T04:30:07.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:07 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:07.385+0000 7f7779de0640 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:30:08.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:08 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:08.017+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:08.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:08 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:08.017+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:08.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:08 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:08.017+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:08.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:08 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:08.250+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:08.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:08 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:08.250+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:08.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:08 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:08.250+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:09.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:09.006+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:09.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:09.006+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:09.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:09 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:09.006+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:09.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:09.292+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:09.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:09.292+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:09.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:09 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:09.292+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:10.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:10 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:10.013+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:10.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:10 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:10.013+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:10.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:10 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:10.013+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:10.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:10 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:10.268+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:10.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:10 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:10.268+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:10.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:10 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:10.268+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:11.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:11.015+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:11.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:11.015+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:11.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:11 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:11.015+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:11.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:11.316+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:11.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:11.316+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:11.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:11 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:11.316+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:12.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:12.010+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:12.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:12.010+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:12.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:12.010+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:12.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:12.313+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:43.203698+0000 front 2024-08-01T04:28:43.203821+0000 (oldest deadline 2024-08-01T04:29:47.203640+0000) 2024-08-01T04:30:12.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:12.313+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:50.104541+0000 front 2024-08-01T04:28:50.104515+0000 (oldest deadline 2024-08-01T04:29:56.004721+0000) 2024-08-01T04:30:12.653 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:12 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10[61667]: 2024-08-01T04:30:12.313+0000 7f77723ee640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:01.806303+0000 front 2024-08-01T04:29:01.806273+0000 (oldest deadline 2024-08-01T04:30:03.506395+0000) 2024-08-01T04:30:13.295 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:13.011+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:13.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:13.011+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:13.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:13 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:13.011+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:13.345 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:13 smithi160 podman[73414]: 2024-08-01 04:30:13.063076111 +0000 UTC m=+5.771442370 container died 545b8eaab26f074ec90f5b4e826577fe785ced4c492999f80fa881b6dc505058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1) 2024-08-01T04:30:13.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:13 smithi160 podman[73414]: 2024-08-01 04:30:13.435504001 +0000 UTC m=+6.143870247 container cleanup 545b8eaab26f074ec90f5b4e826577fe785ced4c492999f80fa881b6dc505058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-08-01T04:30:13.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:13 smithi160 bash[73414]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10 2024-08-01T04:30:14.048 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:13 smithi160 podman[73429]: 2024-08-01 04:30:13.71154648 +0000 UTC m=+0.642838000 container remove 545b8eaab26f074ec90f5b4e826577fe785ced4c492999f80fa881b6dc505058 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True) 2024-08-01T04:30:14.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:14 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:14.005+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:14.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:14 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:14.005+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:14.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:14 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:14.005+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:14.324 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73517]: 2024-08-01 04:30:13.950778243 +0000 UTC m=+0.017493894 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:30:14.324 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73517]: 2024-08-01 04:30:14.053905181 +0000 UTC m=+0.120620838 container create 46f6d0a64a32190b7784857b24f7513f248b6edd3bf9206ed85bf0ae47529545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-01T04:30:14.324 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73517]: 2024-08-01 04:30:14.247961894 +0000 UTC m=+0.314677560 container init 46f6d0a64a32190b7784857b24f7513f248b6edd3bf9206ed85bf0ae47529545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:14.324 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73517]: 2024-08-01 04:30:14.250932854 +0000 UTC m=+0.317648508 container start 46f6d0a64a32190b7784857b24f7513f248b6edd3bf9206ed85bf0ae47529545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-01T04:30:14.324 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73517]: 2024-08-01 04:30:14.322699379 +0000 UTC m=+0.389415056 container attach 46f6d0a64a32190b7784857b24f7513f248b6edd3bf9206ed85bf0ae47529545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS) 2024-08-01T04:30:14.652 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73517]: 2024-08-01 04:30:14.439572024 +0000 UTC m=+0.506287682 container died 46f6d0a64a32190b7784857b24f7513f248b6edd3bf9206ed85bf0ae47529545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:14.982 INFO:journalctl@ceph.osd.10.smithi160.stdout:Aug 01 04:30:14 smithi160 podman[73540]: 2024-08-01 04:30:14.967510287 +0000 UTC m=+0.522384945 container remove 46f6d0a64a32190b7784857b24f7513f248b6edd3bf9206ed85bf0ae47529545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:30:14.987 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.10.service' 2024-08-01T04:30:15.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:15.024 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-01T04:30:15.025 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-01T04:30:15.025 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.11 2024-08-01T04:30:15.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.034+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:15.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.034+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:15.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.034+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:15.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.034+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-08-01T04:29:07.168604+0000 front 2024-08-01T04:29:07.168575+0000 (oldest deadline 2024-08-01T04:30:14.768814+0000) 2024-08-01T04:30:15.297 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 systemd[1]: Stopping Ceph osd.11 for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:15.796 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.317+0000 7fe7e4d6f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:15.796 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.317+0000 7fe7e4d6f640 -1 osd.11 104 *** Got signal Terminated *** 2024-08-01T04:30:15.796 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:15 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:15.317+0000 7fe7e4d6f640 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-01T04:30:16.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:16 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:16.077+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:16.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:16 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:16.077+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:16.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:16 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:16.077+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:16.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:16 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:16.077+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-08-01T04:29:07.168604+0000 front 2024-08-01T04:29:07.168575+0000 (oldest deadline 2024-08-01T04:30:14.768814+0000) 2024-08-01T04:30:17.545 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:17 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:17.123+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:17.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:17 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:17.123+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:17.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:17 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:17.123+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:17.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:17 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:17.123+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-08-01T04:29:07.168604+0000 front 2024-08-01T04:29:07.168575+0000 (oldest deadline 2024-08-01T04:30:14.768814+0000) 2024-08-01T04:30:18.545 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:18 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:18.144+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:18.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:18 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:18.144+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:18.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:18 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:18.144+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:18.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:18 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:18.144+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-08-01T04:29:07.168604+0000 front 2024-08-01T04:29:07.168575+0000 (oldest deadline 2024-08-01T04:30:14.768814+0000) 2024-08-01T04:30:19.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:19 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:19.118+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:19.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:19 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:19.118+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:19.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:19 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:19.118+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:19.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:19 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:19.118+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-08-01T04:29:07.168604+0000 front 2024-08-01T04:29:07.168575+0000 (oldest deadline 2024-08-01T04:30:14.768814+0000) 2024-08-01T04:30:20.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:20 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:20.070+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-08-01T04:28:40.864796+0000 front 2024-08-01T04:28:40.864868+0000 (oldest deadline 2024-08-01T04:29:44.964591+0000) 2024-08-01T04:30:20.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:20 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:20.070+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-08-01T04:28:47.865626+0000 front 2024-08-01T04:28:47.865715+0000 (oldest deadline 2024-08-01T04:29:57.866186+0000) 2024-08-01T04:30:20.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:20 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:20.070+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.163:6804 osd.2 since back 2024-08-01T04:29:00.767358+0000 front 2024-08-01T04:29:00.767587+0000 (oldest deadline 2024-08-01T04:30:06.667558+0000) 2024-08-01T04:30:20.546 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:20 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11[62437]: 2024-08-01T04:30:20.070+0000 7fe7e1385640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-08-01T04:29:07.168604+0000 front 2024-08-01T04:29:07.168575+0000 (oldest deadline 2024-08-01T04:30:14.768814+0000) 2024-08-01T04:30:21.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:21 smithi163 podman[74320]: 2024-08-01 04:30:21.043998308 +0000 UTC m=+5.823910539 container died a5e21f1d2f59d86bac30f5f48747f84182c593a3b41b7b351d07ef8500a307e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-01T04:30:21.670 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:21 smithi163 podman[74320]: 2024-08-01 04:30:21.392803221 +0000 UTC m=+6.172715452 container cleanup a5e21f1d2f59d86bac30f5f48747f84182c593a3b41b7b351d07ef8500a307e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0) 2024-08-01T04:30:21.670 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:21 smithi163 bash[74320]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11 2024-08-01T04:30:21.994 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:21 smithi163 podman[74348]: 2024-08-01 04:30:21.669062026 +0000 UTC m=+0.624719604 container remove a5e21f1d2f59d86bac30f5f48747f84182c593a3b41b7b351d07ef8500a307e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:30:22.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:21 smithi163 podman[74437]: 2024-08-01 04:30:21.895736531 +0000 UTC m=+0.017042692 image pull 93d93c74a2cc6f07ff8c0848a01017c27414830696330be353461f8545597967 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483 2024-08-01T04:30:22.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:22 smithi163 podman[74437]: 2024-08-01 04:30:22.009314263 +0000 UTC m=+0.130620468 container create c186ce3ea3bde48d478c9f60789fd509348296b3edf3efaf33ae6ba1f7741c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1) 2024-08-01T04:30:22.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:22 smithi163 podman[74437]: 2024-08-01 04:30:22.231653645 +0000 UTC m=+0.352959810 container init c186ce3ea3bde48d478c9f60789fd509348296b3edf3efaf33ae6ba1f7741c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-01T04:30:22.296 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:22 smithi163 podman[74437]: 2024-08-01 04:30:22.235199556 +0000 UTC m=+0.356505716 container start c186ce3ea3bde48d478c9f60789fd509348296b3edf3efaf33ae6ba1f7741c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:22.709 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:22 smithi163 podman[74437]: 2024-08-01 04:30:22.314135964 +0000 UTC m=+0.435442138 container attach c186ce3ea3bde48d478c9f60789fd509348296b3edf3efaf33ae6ba1f7741c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2) 2024-08-01T04:30:22.709 INFO:journalctl@ceph.osd.11.smithi163.stdout:Aug 01 04:30:22 smithi163 podman[74437]: 2024-08-01 04:30:22.421010247 +0000 UTC m=+0.542316422 container died c186ce3ea3bde48d478c9f60789fd509348296b3edf3efaf33ae6ba1f7741c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-osd-11-deactivate, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:23.043 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@osd.11.service' 2024-08-01T04:30:23.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:23.081 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-01T04:30:23.081 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-01T04:30:23.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.a 2024-08-01T04:30:23.518 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:23 smithi003 systemd[1]: Stopping Ceph mds.a for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:23.798 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a[72667]: 2024-08-01T04:30:23.733+0000 7fc872600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:23.799 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:23 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a[72667]: 2024-08-01T04:30:23.733+0000 7fc872600640 -1 mds.a *** got signal Terminated *** 2024-08-01T04:30:28.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:27 smithi003 podman[106241]: 2024-08-01 04:30:27.760601899 +0000 UTC m=+4.164281176 container died e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-01T04:30:28.548 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:28 smithi003 podman[106241]: 2024-08-01 04:30:28.258736281 +0000 UTC m=+4.662415581 container cleanup e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, io.buildah.version=1.36.0) 2024-08-01T04:30:28.549 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:28 smithi003 bash[106241]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a 2024-08-01T04:30:28.995 INFO:journalctl@ceph.mds.a.smithi003.stdout:Aug 01 04:30:28 smithi003 podman[106266]: 2024-08-01 04:30:28.677262062 +0000 UTC m=+0.899645607 container remove e37918a6ea2bde3c6de6d51295c779f72732cdc2589a4682839fd57ac7d84d5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-01T04:30:29.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.a.service' 2024-08-01T04:30:29.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:29.148 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-01T04:30:29.148 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-01T04:30:29.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.d 2024-08-01T04:30:29.549 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:29 smithi003 systemd[1]: Stopping Ceph mds.d for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:30.048 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:29 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d[71554]: 2024-08-01T04:30:29.777+0000 7ff139200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:30.049 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:29 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d[71554]: 2024-08-01T04:30:29.777+0000 7ff139200640 -1 mds.d *** got signal Terminated *** 2024-08-01T04:30:34.049 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:33 smithi003 podman[106375]: 2024-08-01 04:30:33.722285482 +0000 UTC m=+4.080206486 container died aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724) 2024-08-01T04:30:34.548 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:34 smithi003 podman[106375]: 2024-08-01 04:30:34.329786007 +0000 UTC m=+4.687707014 container cleanup aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-01T04:30:34.548 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:34 smithi003 bash[106375]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d 2024-08-01T04:30:34.936 INFO:journalctl@ceph.mds.d.smithi003.stdout:Aug 01 04:30:34 smithi003 podman[106387]: 2024-08-01 04:30:34.639816713 +0000 UTC m=+0.898593831 container remove aeb472cdb36516186c76c427b844cddfc85b5d2dee3cbd22e37877756295edf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, ceph=True) 2024-08-01T04:30:34.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.d.service' 2024-08-01T04:30:35.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:35.078 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-01T04:30:35.078 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-01T04:30:35.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.g 2024-08-01T04:30:35.302 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:35 smithi003 systemd[1]: Stopping Ceph mds.g for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:36.049 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:35 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-g[72309]: 2024-08-01T04:30:35.735+0000 7f7d6cc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:36.049 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:35 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-g[72309]: 2024-08-01T04:30:35.735+0000 7f7d6cc00640 -1 mds.g *** got signal Terminated *** 2024-08-01T04:30:39.273 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:38 smithi003 podman[106496]: 2024-08-01 04:30:38.997098885 +0000 UTC m=+3.419421766 container died d570236853748e20f71e6b2bfb1eb94fef34dec90f5db49461ff217634e59add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:30:39.548 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:39 smithi003 podman[106496]: 2024-08-01 04:30:39.458967525 +0000 UTC m=+3.881290406 container cleanup d570236853748e20f71e6b2bfb1eb94fef34dec90f5db49461ff217634e59add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-08-01T04:30:39.548 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:39 smithi003 bash[106496]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-g 2024-08-01T04:30:40.048 INFO:journalctl@ceph.mds.g.smithi003.stdout:Aug 01 04:30:39 smithi003 podman[106507]: 2024-08-01 04:30:39.727259725 +0000 UTC m=+0.713453298 container remove d570236853748e20f71e6b2bfb1eb94fef34dec90f5db49461ff217634e59add (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:30:40.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.g.service' 2024-08-01T04:30:40.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:40.402 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-01T04:30:40.402 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-01T04:30:40.402 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.j 2024-08-01T04:30:40.548 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:40 smithi003 systemd[1]: Stopping Ceph mds.j for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:41.298 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:40 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j[71143]: 2024-08-01T04:30:40.969+0000 7f0046a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:41.299 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:40 smithi003 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j[71143]: 2024-08-01T04:30:40.969+0000 7f0046a00640 -1 mds.j *** got signal Terminated *** 2024-08-01T04:30:45.293 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:44 smithi003 podman[106615]: 2024-08-01 04:30:44.985369557 +0000 UTC m=+4.129289327 container died cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-01T04:30:45.548 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:45 smithi003 podman[106615]: 2024-08-01 04:30:45.396325367 +0000 UTC m=+4.540245103 container cleanup cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j, org.label-schema.build-date=20240724, GIT_CLEAN=True, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:30:45.549 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:45 smithi003 bash[106615]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j 2024-08-01T04:30:46.048 INFO:journalctl@ceph.mds.j.smithi003.stdout:Aug 01 04:30:45 smithi003 podman[106628]: 2024-08-01 04:30:45.663821495 +0000 UTC m=+0.658059256 container remove cf5e3f2346f84388c920510f984457ab52563d75df241ac46090c1acac3cd374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-01T04:30:46.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.j.service' 2024-08-01T04:30:46.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:46.239 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-01T04:30:46.239 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-01T04:30:46.239 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.b 2024-08-01T04:30:46.543 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:46 smithi160 systemd[1]: Stopping Ceph mds.b for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:46.801 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:46 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: 2024-08-01T04:30:46.541+0000 7fd13abb7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:46.802 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:46 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b[67282]: 2024-08-01T04:30:46.541+0000 7fd13abb7640 -1 mds.b *** got signal Terminated *** 2024-08-01T04:30:48.034 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:47 smithi160 podman[73620]: 2024-08-01 04:30:47.791825703 +0000 UTC m=+1.355765754 container died 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:30:48.335 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:48 smithi160 podman[73620]: 2024-08-01 04:30:48.133207903 +0000 UTC m=+1.697147948 container cleanup 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-f9911e1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-01T04:30:48.335 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:48 smithi160 bash[73620]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b 2024-08-01T04:30:48.562 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.b.service' 2024-08-01T04:30:48.585 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:48 smithi160 podman[73643]: 2024-08-01 04:30:48.334174366 +0000 UTC m=+0.539540503 container remove 03633524b63685218ae83cb73d0d59090ad37aca24438af8044f870d9aa57b38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-08-01T04:30:48.586 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:48 smithi160 systemd[1]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.b.service: Deactivated successfully. 2024-08-01T04:30:48.586 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:48 smithi160 systemd[1]: Stopped Ceph mds.b for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968. 2024-08-01T04:30:48.586 INFO:journalctl@ceph.mds.b.smithi160.stdout:Aug 01 04:30:48 smithi160 systemd[1]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.b.service: Consumed 26.929s CPU time. 2024-08-01T04:30:48.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:48.598 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-01T04:30:48.598 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-01T04:30:48.598 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.e 2024-08-01T04:30:48.902 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:48 smithi160 systemd[1]: Stopping Ceph mds.e for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:49.402 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:48 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e[66665]: 2024-08-01T04:30:48.919+0000 7fc1bd15a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:49.403 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:48 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e[66665]: 2024-08-01T04:30:48.919+0000 7fc1bd15a640 -1 mds.e *** got signal Terminated *** 2024-08-01T04:30:52.087 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:51 smithi160 podman[73754]: 2024-08-01 04:30:51.769098306 +0000 UTC m=+2.945947603 container died 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:30:52.088 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:52 smithi160 podman[73754]: 2024-08-01 04:30:52.086737994 +0000 UTC m=+3.263587277 container cleanup 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1) 2024-08-01T04:30:52.402 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:52 smithi160 bash[73754]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e 2024-08-01T04:30:52.402 INFO:journalctl@ceph.mds.e.smithi160.stdout:Aug 01 04:30:52 smithi160 podman[73766]: 2024-08-01 04:30:52.262756278 +0000 UTC m=+0.492880010 container remove 3ef621ec6fb5fc2833802061a1e7cc09b7f65d1c52689a5ef6269ecbbec8c99f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-01T04:30:52.522 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.e.service' 2024-08-01T04:30:52.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:52.555 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-01T04:30:52.556 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-01T04:30:52.556 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.h 2024-08-01T04:30:52.866 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:52 smithi160 systemd[1]: Stopping Ceph mds.h for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:53.152 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:52 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h[66968]: 2024-08-01T04:30:52.865+0000 7f5ff45bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:53.153 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:52 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h[66968]: 2024-08-01T04:30:52.865+0000 7f5ff45bc640 -1 mds.h *** got signal Terminated *** 2024-08-01T04:30:56.091 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:55 smithi160 podman[73875]: 2024-08-01 04:30:55.774665827 +0000 UTC m=+3.000605608 container died 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:30:56.091 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:56 smithi160 podman[73875]: 2024-08-01 04:30:56.090066128 +0000 UTC m=+3.316005901 container cleanup 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0) 2024-08-01T04:30:56.402 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:56 smithi160 bash[73875]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h 2024-08-01T04:30:56.402 INFO:journalctl@ceph.mds.h.smithi160.stdout:Aug 01 04:30:56 smithi160 podman[73888]: 2024-08-01 04:30:56.282920503 +0000 UTC m=+0.491189700 container remove 05287f9082e1244430f9116dc6d6d5cce072a00886f0a486b4543a552adcd9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-h, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-f9911e1, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-01T04:30:56.425 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.h.service' 2024-08-01T04:30:56.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:30:56.457 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-01T04:30:56.457 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-01T04:30:56.458 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.k 2024-08-01T04:30:56.786 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:30:56 smithi160 systemd[1]: Stopping Ceph mds.k for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:30:57.152 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:30:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k[66352]: 2024-08-01T04:30:56.784+0000 7f509be7c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:30:57.152 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:30:56 smithi160 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k[66352]: 2024-08-01T04:30:56.784+0000 7f509be7c640 -1 mds.k *** got signal Terminated *** 2024-08-01T04:31:01.152 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:31:00 smithi160 podman[73993]: 2024-08-01 04:31:00.797799575 +0000 UTC m=+4.116820848 container died 83e17d5f6434875f320fd02adea1267fab889af745b59581aec5b666d1d344ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:31:01.652 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:31:01 smithi160 podman[73993]: 2024-08-01 04:31:01.426734438 +0000 UTC m=+4.745755708 container cleanup 83e17d5f6434875f320fd02adea1267fab889af745b59581aec5b666d1d344ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-01T04:31:01.652 INFO:journalctl@ceph.mds.k.smithi160.stdout:Aug 01 04:31:01 smithi160 bash[73993]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-k 2024-08-01T04:31:01.802 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.k.service' 2024-08-01T04:31:01.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:31:01.834 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-01T04:31:01.834 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-01T04:31:01.834 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.c 2024-08-01T04:31:02.135 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:31:01 smithi163 systemd[1]: Stopping Ceph mds.c for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:31:02.545 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:31:02 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c[67542]: 2024-08-01T04:31:02.133+0000 7fb8476ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:31:02.546 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:31:02 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c[67542]: 2024-08-01T04:31:02.133+0000 7fb8476ec640 -1 mds.c *** got signal Terminated *** 2024-08-01T04:31:04.076 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:31:03 smithi163 podman[74542]: 2024-08-01 04:31:03.798879939 +0000 UTC m=+1.775660093 container died 9eae5ad1084f43f65e7b0fa8f7eb122d3c89a079f5d328bb32c5e1a32efda9e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-f9911e1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-01T04:31:04.419 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:31:04 smithi163 podman[74542]: 2024-08-01 04:31:04.183472648 +0000 UTC m=+2.160252792 container cleanup 9eae5ad1084f43f65e7b0fa8f7eb122d3c89a079f5d328bb32c5e1a32efda9e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-f9911e1, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-01T04:31:04.419 INFO:journalctl@ceph.mds.c.smithi163.stdout:Aug 01 04:31:04 smithi163 bash[74542]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-c 2024-08-01T04:31:04.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.c.service' 2024-08-01T04:31:04.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:31:04.587 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-01T04:31:04.587 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-01T04:31:04.587 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.f 2024-08-01T04:31:04.795 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:31:04 smithi163 systemd[1]: Stopping Ceph mds.f for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:31:05.295 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:31:04 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f[67241]: 2024-08-01T04:31:04.912+0000 7ff691387640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:31:05.296 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:31:04 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f[67241]: 2024-08-01T04:31:04.912+0000 7ff691387640 -1 mds.f *** got signal Terminated *** 2024-08-01T04:31:07.045 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:31:06 smithi163 podman[74663]: 2024-08-01 04:31:06.786711206 +0000 UTC m=+1.983965346 container died 6b97cc6642de49d9583fd6dbcd8171ee91de6ad49e29aa304aa85c8955d147ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-f9911e1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, ceph=True, org.label-schema.vendor=CentOS) 2024-08-01T04:31:07.439 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:31:07 smithi163 podman[74663]: 2024-08-01 04:31:07.137838849 +0000 UTC m=+2.335092978 container cleanup 6b97cc6642de49d9583fd6dbcd8171ee91de6ad49e29aa304aa85c8955d147ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-01T04:31:07.440 INFO:journalctl@ceph.mds.f.smithi163.stdout:Aug 01 04:31:07 smithi163 bash[74663]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-f 2024-08-01T04:31:07.563 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.f.service' 2024-08-01T04:31:07.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:31:07.594 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-01T04:31:07.594 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-01T04:31:07.595 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.i 2024-08-01T04:31:07.795 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:31:07 smithi163 systemd[1]: Stopping Ceph mds.i for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:31:08.295 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:31:07 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i[67845]: 2024-08-01T04:31:07.908+0000 7f211d2b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:31:08.296 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:31:07 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i[67845]: 2024-08-01T04:31:07.908+0000 7f211d2b3640 -1 mds.i *** got signal Terminated *** 2024-08-01T04:31:12.093 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:31:11 smithi163 podman[74780]: 2024-08-01 04:31:11.801949946 +0000 UTC m=+3.990927041 container died 7854fce8c9941a5293b520969ed9e33c82131d66581f8cea35715882233d9c90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, RELEASE=squid-f9911e1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:31:12.444 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:31:12 smithi163 podman[74780]: 2024-08-01 04:31:12.166975606 +0000 UTC m=+4.355952711 container cleanup 7854fce8c9941a5293b520969ed9e33c82131d66581f8cea35715882233d9c90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-01T04:31:12.444 INFO:journalctl@ceph.mds.i.smithi163.stdout:Aug 01 04:31:12 smithi163 bash[74780]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-i 2024-08-01T04:31:12.594 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.i.service' 2024-08-01T04:31:12.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:31:12.626 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-01T04:31:12.626 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-01T04:31:12.626 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.l 2024-08-01T04:31:13.046 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:31:12 smithi163 systemd[1]: Stopping Ceph mds.l for 2f6d75a6-4fbb-11ef-bcc9-c7b262605968... 2024-08-01T04:31:13.046 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:31:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l[66933]: 2024-08-01T04:31:12.945+0000 7f7c99208640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-01T04:31:13.046 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:31:12 smithi163 ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l[66933]: 2024-08-01T04:31:12.945+0000 7f7c99208640 -1 mds.l *** got signal Terminated *** 2024-08-01T04:31:14.731 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:31:14 smithi163 podman[74903]: 2024-08-01 04:31:14.451422339 +0000 UTC m=+1.606253370 container died 5a666cacb5926c1fed6216a11f99dcf90042866a01b3405c44722b4dbc40e20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-f9911e1, GIT_CLEAN=True, org.label-schema.build-date=20240724, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-01T04:31:15.045 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:31:14 smithi163 podman[74903]: 2024-08-01 04:31:14.821392997 +0000 UTC m=+1.976224027 container cleanup 5a666cacb5926c1fed6216a11f99dcf90042866a01b3405c44722b4dbc40e20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0190f631ac6750a6092012483b23089df6a4e4f17cd9c82e8b6d0c32325e1483, name=ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-f9911e1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-01T04:31:15.046 INFO:journalctl@ceph.mds.l.smithi163.stdout:Aug 01 04:31:14 smithi163 bash[74903]: ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968-mds-l 2024-08-01T04:31:15.210 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2f6d75a6-4fbb-11ef-bcc9-c7b262605968@mds.l.service' 2024-08-01T04:31:15.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-01T04:31:15.241 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-01T04:31:15.241 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --force --keep-logs 2024-08-01T04:31:15.591 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:31:20.797 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --force --keep-logs 2024-08-01T04:31:21.002 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:31:23.546 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --force --keep-logs 2024-08-01T04:31:23.748 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:31:26.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:31:26.380 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:31:26.412 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-01T04:31:26.438 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-01T04:31:26.441 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi003/crash 2024-08-01T04:31:26.442 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash -- . 2024-08-01T04:31:26.542 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash: Cannot open: No such file or directory 2024-08-01T04:31:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2024-08-01T04:31:26.548 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi160/crash 2024-08-01T04:31:26.548 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash -- . 2024-08-01T04:31:26.593 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash: Cannot open: No such file or directory 2024-08-01T04:31:26.594 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-08-01T04:31:26.595 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi163/crash 2024-08-01T04:31:26.596 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash -- . 2024-08-01T04:31:26.652 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/crash: Cannot open: No such file or directory 2024-08-01T04:31:26.653 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2024-08-01T04:31:26.654 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-01T04:31:26.654 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-01T04:31:26.834 INFO:tasks.cephadm:Compressing logs... 2024-08-01T04:31:26.835 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-01T04:31:26.840 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-01T04:31:26.842 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-01T04:31:26.865 INFO:teuthology.orchestra.run.smithi163.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-01T04:31:26.872 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-01T04:31:26.872 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log 2024-08-01T04:31:26.876 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log 2024-08-01T04:31:26.876 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.7.log 2024-08-01T04:31:26.876 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.b.log 2024-08-01T04:31:26.876 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.4.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.1.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.y.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.4.log: /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.h.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.10.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.e.log 2024-08-01T04:31:26.877 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log 2024-08-01T04:31:26.878 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose --/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.e.log: /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.b.log 2024-08-01T04:31:26.880 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.k.log 2024-08-01T04:31:26.880 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-01T04:31:26.883 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log 2024-08-01T04:31:26.884 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.2.log 2024-08-01T04:31:26.884 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.f.log 2024-08-01T04:31:26.884 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log 2024-08-01T04:31:26.884 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.5.log 2024-08-01T04:31:26.884 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.c.log/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log: 2024-08-01T04:31:26.884 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.c.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.z.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.i.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.11.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.8.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log 2024-08-01T04:31:26.885 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.l.log 2024-08-01T04:31:26.886 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-01T04:31:26.925 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-01T04:31:26.925 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log 2024-08-01T04:31:26.926 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log 2024-08-01T04:31:26.927 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.g.log 2024-08-01T04:31:26.928 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.j.log 2024-08-01T04:31:26.930 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.3.log 2024-08-01T04:31:26.931 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.j.log: 86.8% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log.gz 2024-08-01T04:31:26.931 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.0.log 2024-08-01T04:31:26.932 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.x.log 2024-08-01T04:31:26.934 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.a.log 2024-08-01T04:31:26.935 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.j.log.gz 2024-08-01T04:31:26.935 INFO:teuthology.orchestra.run.smithi003.stderr: 94.4% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.g.log.gzgzip 2024-08-01T04:31:26.935 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.6.log 2024-08-01T04:31:26.936 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log 2024-08-01T04:31:26.937 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.a.log 2024-08-01T04:31:26.938 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log 2024-08-01T04:31:26.939 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.d.log 2024-08-01T04:31:26.940 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.9.log 2024-08-01T04:31:26.941 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-01T04:31:26.945 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.d.log.gz 2024-08-01T04:31:26.994 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 82.1% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log.gz 2024-08-01T04:31:26.995 INFO:teuthology.orchestra.run.smithi160.stderr: 90.3% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log.gz 2024-08-01T04:31:27.025 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log.gz 2024-08-01T04:31:27.034 INFO:teuthology.orchestra.run.smithi163.stderr: 94.6% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.l.log.gz 2024-08-01T04:31:27.047 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.z.log.gz 2024-08-01T04:31:27.063 INFO:teuthology.orchestra.run.smithi003.stderr: 84.6% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log.gz 2024-08-01T04:31:27.131 INFO:teuthology.orchestra.run.smithi160.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-01T04:31:27.154 INFO:teuthology.orchestra.run.smithi160.stderr: 92.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.y.log.gz 2024-08-01T04:31:27.173 INFO:teuthology.orchestra.run.smithi003.stderr: 92.5% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.a.log.gz 2024-08-01T04:31:27.190 INFO:teuthology.orchestra.run.smithi160.stderr: 86.8% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log.gz 2024-08-01T04:31:27.194 INFO:teuthology.orchestra.run.smithi163.stderr: 82.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.cephadm.log.gz 2024-08-01T04:31:27.198 INFO:teuthology.orchestra.run.smithi163.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-01T04:31:27.244 INFO:teuthology.orchestra.run.smithi160.stderr: 94.5% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.k.log.gz 2024-08-01T04:31:27.261 INFO:teuthology.orchestra.run.smithi160.stderr: 91.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log.gz 2024-08-01T04:31:27.318 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log.gz 2024-08-01T04:31:27.367 INFO:teuthology.orchestra.run.smithi163.stderr: 90.4% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.audit.log.gz 2024-08-01T04:31:27.388 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-volume.log.gz 2024-08-01T04:31:27.433 INFO:teuthology.orchestra.run.smithi163.stderr: 94.6% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.f.log.gz 2024-08-01T04:31:27.568 INFO:teuthology.orchestra.run.smithi160.stderr: 93.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.h.log.gz 2024-08-01T04:31:27.649 INFO:teuthology.orchestra.run.smithi163.stderr: 86.8% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph.log.gz 2024-08-01T04:31:27.770 INFO:teuthology.orchestra.run.smithi003.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-01T04:31:27.838 INFO:teuthology.orchestra.run.smithi160.stderr: 94.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.e.log.gz 2024-08-01T04:31:27.887 INFO:teuthology.orchestra.run.smithi163.stderr: 94.4% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.c.log.gz 2024-08-01T04:31:28.608 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mgr.x.log.gz 2024-08-01T04:31:32.106 INFO:teuthology.orchestra.run.smithi163.stderr: 91.5% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.c.log.gz 2024-08-01T04:31:32.583 INFO:teuthology.orchestra.run.smithi003.stderr: 90.4% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.a.log.gz 2024-08-01T04:31:37.124 INFO:teuthology.orchestra.run.smithi160.stderr: 91.5% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mon.b.log.gz 2024-08-01T04:31:41.190 INFO:teuthology.orchestra.run.smithi163.stderr: 95.8% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.i.log.gz 2024-08-01T04:32:31.343 INFO:teuthology.orchestra.run.smithi160.stderr: 94.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-mds.b.log.gz 2024-08-01T04:33:23.756 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.11.log.gz 2024-08-01T04:34:42.343 INFO:teuthology.orchestra.run.smithi160.stderr: 92.3% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.10.log.gz 2024-08-01T04:35:13.837 INFO:teuthology.orchestra.run.smithi160.stderr: 92.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.7.log.gz 2024-08-01T04:38:15.255 INFO:teuthology.orchestra.run.smithi160.stderr: 92.3% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.4.log.gz 2024-08-01T04:38:44.925 INFO:teuthology.orchestra.run.smithi160.stderr: 92.1% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.1.log.gz 2024-08-01T04:38:44.926 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-08-01T04:38:44.926 INFO:teuthology.orchestra.run.smithi160.stderr:real 7m18.069s 2024-08-01T04:38:44.926 INFO:teuthology.orchestra.run.smithi160.stderr:user 6m49.044s 2024-08-01T04:38:44.927 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m15.839s 2024-08-01T04:40:33.440 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.8.log.gz 2024-08-01T04:41:29.790 INFO:teuthology.orchestra.run.smithi003.stderr: 92.3% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.9.log.gz 2024-08-01T04:42:42.221 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.2.log.gz 2024-08-01T04:42:43.066 INFO:teuthology.orchestra.run.smithi003.stderr: 92.1% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.6.log.gz 2024-08-01T04:42:57.627 INFO:teuthology.orchestra.run.smithi003.stderr: 92.0% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.3.log.gz 2024-08-01T04:43:00.342 INFO:teuthology.orchestra.run.smithi003.stderr: 92.1% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.0.log.gz 2024-08-01T04:43:00.345 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-08-01T04:43:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:real 11m33.461s 2024-08-01T04:43:00.345 INFO:teuthology.orchestra.run.smithi003.stderr:user 16m54.910s 2024-08-01T04:43:00.346 INFO:teuthology.orchestra.run.smithi003.stderr:sys 1m58.926s 2024-08-01T04:43:01.004 INFO:teuthology.orchestra.run.smithi163.stderr: 92.2% -- replaced with /var/log/ceph/2f6d75a6-4fbb-11ef-bcc9-c7b262605968/ceph-osd.5.log.gz 2024-08-01T04:43:01.007 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-08-01T04:43:01.007 INFO:teuthology.orchestra.run.smithi163.stderr:real 11m34.150s 2024-08-01T04:43:01.007 INFO:teuthology.orchestra.run.smithi163.stderr:user 6m53.372s 2024-08-01T04:43:01.007 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m16.961s 2024-08-01T04:43:01.008 INFO:tasks.cephadm:Archiving logs... 2024-08-01T04:43:01.009 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi003/log 2024-08-01T04:43:01.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-01T04:44:07.529 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi160/log 2024-08-01T04:44:07.530 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-01T04:45:13.166 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi163/log 2024-08-01T04:45:13.169 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-01T04:46:18.273 INFO:tasks.cephadm:Removing cluster... 2024-08-01T04:46:18.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --force 2024-08-01T04:46:19.038 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:46:20.965 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --force 2024-08-01T04:46:21.170 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:46:21.879 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 --force 2024-08-01T04:46:22.293 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 2f6d75a6-4fbb-11ef-bcc9-c7b262605968 2024-08-01T04:46:23.005 INFO:tasks.cephadm:Removing cephadm ... 2024-08-01T04:46:23.006 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-01T04:46:23.078 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-01T04:46:23.097 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-01T04:46:23.113 INFO:tasks.cephadm:Teardown complete 2024-08-01T04:46:23.113 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-01T04:46:23.124 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-01T04:46:23.125 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-01T04:46:23.132 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-01T04:46:23.140 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-01T04:46:23.325 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-01T04:46:23.336 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-01T04:46:23.337 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-01T04:46:23.345 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-01T04:46:23.347 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-01T04:46:23.362 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-08-01T04:46:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-08-01T04:46:23.379 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-01T04:46:23.379 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-08-01T04:46:23.379 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -515us[ -506us] +/- 64ms 2024-08-01T04:46:23.380 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -442us[ -442us] +/- 71ms 2024-08-01T04:46:23.380 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +1150us[+1150us] +/- 66ms 2024-08-01T04:46:23.380 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:46:23.381 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-01T04:46:23.382 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-08-01T04:46:23.382 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 -510us[ -510us] +/- 64ms 2024-08-01T04:46:23.382 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 -444us[ -444us] +/- 71ms 2024-08-01T04:46:23.382 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +1141us[+1142us] +/- 66ms 2024-08-01T04:46:23.382 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:46:23.395 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-08-01T04:46:23.525 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-01T04:46:23.526 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-08-01T04:46:23.526 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 7 377 22 -420us[ -418us] +/- 64ms 2024-08-01T04:46:23.526 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 21 -344us[ -344us] +/- 71ms 2024-08-01T04:46:23.526 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 23 +1261us[+1263us] +/- 66ms 2024-08-01T04:46:23.527 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-01T04:46:23.529 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-01T04:46:23.543 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-01T04:46:23.544 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-01T04:46:23.555 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-01T04:46:23.867 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-01T04:46:23.953 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-01T04:46:24.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-01T04:46:24.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:46:24.146 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-01T04:46:24.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:46:24.176 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-01T04:46:24.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:46:24.204 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-01T04:46:24.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-01T04:46:24.226 INFO:teuthology.task.internal:Duration was 2998.028690 seconds 2024-08-01T04:46:24.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-01T04:46:24.238 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-01T04:46:24.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-01T04:46:24.246 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-01T04:46:24.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-01T04:46:24.325 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-01T04:46:24.336 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-01T04:46:24.407 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-01T04:46:24.697 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-01T04:46:24.698 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-08-01T04:46:24.698 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-01T04:46:24.782 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-08-01T04:46:24.782 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-01T04:46:24.815 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-08-01T04:46:24.815 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-01T04:46:24.838 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-01T04:46:24.838 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-01T04:46:24.846 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-01T04:46:24.858 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-01T04:46:24.928 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-01T04:46:24.929 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-01T04:46:24.934 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-01T04:46:24.936 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-01T04:46:25.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-01T04:46:26.009 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-01T04:46:26.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-01T04:46:26.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-01T04:46:26.159 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-01T04:46:26.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-01T04:46:26.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-01T04:46:26.205 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-01T04:46:26.208 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-01T04:46:26.236 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-08-01T04:46:26.274 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-08-01T04:46:26.318 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-08-01T04:46:26.369 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-01T04:46:26.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:46:26.421 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-01T04:46:26.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:46:26.439 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-01T04:46:26.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-01T04:46:26.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-01T04:46:26.467 INFO:teuthology.task.internal:Transferring archived files... 2024-08-01T04:46:26.468 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi003 2024-08-01T04:46:26.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-01T04:46:26.590 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi160 2024-08-01T04:46:26.590 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-01T04:46:26.629 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-07-31_21:24:16-fs-squid-distro-default-smithi/7829148/remote/smithi163 2024-08-01T04:46:26.630 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-01T04:46:26.667 INFO:teuthology.task.internal:Removing archive directory... 2024-08-01T04:46:26.667 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-01T04:46:26.675 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-01T04:46:26.678 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-01T04:46:26.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-01T04:46:26.743 INFO:teuthology.task.internal:Not uploading archives. 2024-08-01T04:46:26.743 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-01T04:46:26.754 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-01T04:46:26.754 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-01T04:46:26.762 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-01T04:46:26.765 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-01T04:46:26.779 INFO:teuthology.orchestra.run.smithi160.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 04:46 /home/ubuntu/cephtest 2024-08-01T04:46:26.781 INFO:teuthology.orchestra.run.smithi163.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 04:46 /home/ubuntu/cephtest 2024-08-01T04:46:26.814 INFO:teuthology.orchestra.run.smithi003.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 1 04:46 /home/ubuntu/cephtest 2024-08-01T04:46:26.817 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-01T04:46:26.830 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-01T04:46:26.860 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 34096973aec0bfbe2bb86f34701acc6cd4133fd9 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 2998.0286903381348 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi003 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d99c9769be1521395979e8c23d9073adbf1ff43f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=035c9d7ff720453d974831aa65054c94 status: fail success: false 2024-08-01T04:46:26.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-01T04:46:26.975 INFO:teuthology.run:FAIL